-- Logs begin at Wed 2022-09-14 19:01:16 UTC, end at Wed 2022-09-14 20:18:53 UTC. -- Sep 14 19:32:37.428677 dv-cinder-839768-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 14 19:32:37.439859 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 14 19:32:37.442171 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 14 19:32:37 2022] *** Sep 14 19:32:37.442171 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 14 19:32:37.442171 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 14 19:32:37.442171 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: nodename: dv-cinder-839768-8-iscsi Sep 14 19:32:37.442171 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: machine: x86_64 Sep 14 19:32:37.442171 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: clock source: unix Sep 14 19:32:37.442171 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: pcre jit disabled Sep 14 19:32:37.442171 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: detected number of CPU cores: 4 Sep 14 19:32:37.442171 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: current working directory: / Sep 14 19:32:37.442171 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: detected binary path: /usr/bin/uwsgi-core Sep 14 19:32:37.442171 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: your processes number limit is 47680 Sep 14 19:32:37.442171 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: your memory page size is 4096 bytes Sep 14 19:32:37.442944 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: detected max file descriptor number: 2048 Sep 14 19:32:37.442944 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: lock engine: pthread robust mutexes Sep 14 19:32:37.442944 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: thunder lock: enabled Sep 14 19:32:37.442944 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 14 19:32:37.442944 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 14 19:32:37.480722 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: Python main interpreter initialized at 0x55c7956619c0 Sep 14 19:32:37.480722 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: python threads support enabled Sep 14 19:32:37.480722 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: your server socket listen backlog is limited to 100 connections Sep 14 19:32:37.480722 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: your mercy for graceful operations on workers is 90 seconds Sep 14 19:32:37.481057 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: mapped 403077 bytes (393 KB) for 2 cores Sep 14 19:32:37.481130 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: *** Operational MODE: preforking *** Sep 14 19:32:37.481400 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: *** uWSGI is running in multiple interpreter mode *** Sep 14 19:32:37.481400 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: spawned uWSGI master process (pid: 93413) Sep 14 19:32:37.481598 dv-cinder-839768-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 14 19:32:37.482462 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: spawned uWSGI worker 1 (pid: 93426, cores: 1) Sep 14 19:32:37.482462 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: spawned uWSGI worker 2 (pid: 93427, cores: 1) Sep 14 19:32:37.482612 dv-cinder-839768-8-iscsi devstack@keystone.service[93413]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 14 19:32:41.810823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 19:32:41.810823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: from cryptography.utils import int_from_bytes Sep 14 19:32:41.811605 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 19:32:41.811605 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: from cryptography.utils import int_from_bytes Sep 14 19:32:41.813639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 19:32:41.813639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: from cryptography.utils import int_from_bytes Sep 14 19:32:41.814136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 19:32:41.814136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: from cryptography.utils import int_from_bytes Sep 14 19:32:41.867862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 14 19:32:41.868042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 14 19:32:41.868120 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 14 19:32:41.868283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] command line args: [] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 14 19:32:41.868368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 14 19:32:41.868437 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] command line args: [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 14 19:32:41.868499 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 14 19:32:41.868653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ================================================================================ {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 14 19:32:41.868744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 14 19:32:41.868905 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] admin_token = **** {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.868991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ================================================================================ {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 14 19:32:41.869152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] config_dir = [] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.869232 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] admin_token = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.869333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.869419 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] config_dir = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.869519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] config_source = [] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.869597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.869663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] debug = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.869771 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] config_source = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.869894 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.870147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] debug = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.870147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.870286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.870354 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.870422 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.870623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.870711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.871684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.871786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.871850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.871918 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] list_limit = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.871918 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] log_config_append = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] list_limit = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872155 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] log_config_append = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] log_dir = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] log_file = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] log_dir = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] log_file = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872509 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872639 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872698 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.872989 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.873055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.873115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.873173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.873236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.873709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.873918 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.874107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.874292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.874516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.875227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.875453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.875624 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.875825 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.876019 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.876019 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.876387 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.876620 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.876788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.876983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.877171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.877362 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.877558 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.877558 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] publish_errors = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.877940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.42/identity {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.877940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] publish_errors = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.878418 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.878614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.878812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.878812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.879123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.879123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.879990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.880145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.880282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.880282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] standard_threads = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.880563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.880696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.880824 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] standard_threads = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.880968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.881098 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.881224 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.881359 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.881479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] use_journal = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.881603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.881752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] use_json = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.881910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] use_journal = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.882042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] use_stderr = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.882167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] use_json = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.882297 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] use_syslog = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.882426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] use_stderr = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.882551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.882761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] use_syslog = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.882897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.883022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 19:32:41.883146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.883275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.883405 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.883537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.883668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.883815 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.883936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.884059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.884183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.884315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.884440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.884573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.884697 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.884826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.884952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.885084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.885214 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.885338 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.885461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.885584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.885731 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.885896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.886021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.886145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.886268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.886401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.886534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.886666 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.886796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.886921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.887047 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.887184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.887316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.887440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.887569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.887693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.887817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.887939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.888085 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.888218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.888343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.888537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.888667 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.888797 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.889001 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.889128 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.889259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.889388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.889512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.889635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.889767 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.889948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.890106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.890230 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.890356 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.890480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.890614 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.890745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.890869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.891022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.891142 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.891266 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.891400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.891524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.891761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.891836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.891903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.891968 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.892035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.892095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.892153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.892215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.892274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.892274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.892388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.892448 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.892505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.892567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.892736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.892991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.893118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.893246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.893368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.893488 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.893609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.893736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.893901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.894068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.894147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.894263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.894345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.894425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.894549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.894626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.894747 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.894823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.894937 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.895011 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.895124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.895207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.895279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.895394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.895475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.895547 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.895661 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.895744 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.895859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.895934 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.896053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.896129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.896245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.896328 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.896401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.896521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.896596 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.896732 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.896814 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.896895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.897075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.897191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.897265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.897378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.897462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.897543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.897616 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.897687 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.897762 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.897891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.897966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.898045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.898117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.898187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.898257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.898327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.898396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.898473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.898545 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.898615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.898685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.898760 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.898831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.898978 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.899053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.899123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.899239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.899313 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.899384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.899463 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.899536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.899612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.899683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.899753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.899823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.899904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.899988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.900059 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.900129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.900199 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.900274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.900351 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.900429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.900501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.900576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.900647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.900717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.900787 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.900865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.900937 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.901006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.901076 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.901146 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.901216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.901296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.901368 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.901438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.901513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.901590 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.901661 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.901739 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.901841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.901841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.901972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.902043 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.902122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.902203 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.902275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.902345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.902414 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.902484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.902553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.902631 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.902703 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.902772 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.902842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.902911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.902980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.903060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.903132 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.903248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.903323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.903400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.903472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.903550 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.903622 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.903692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.903768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.903839 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.903909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.904036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.904112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.904182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] auth.external = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.904251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.904361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.904442 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.904520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.904592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.904674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.904747 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] auth.external = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.904818 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] auth.password = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.904888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.904959 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] auth.token = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.905029 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.905108 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.905180 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.905251 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.905322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] auth.password = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.905392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.905462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] auth.token = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.905541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.905612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.905683 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.905807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.906005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.906087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.906237 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.906322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.906471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] credential.caching = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.906552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.906715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.906794 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.906961 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.907050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.907199 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] credential.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.907278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.907438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.907517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.907680 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.907760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.907912 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.907998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.908155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.908235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.908404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.908485 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.908638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.908717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.908886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.908966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.909147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.909228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.909391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.909471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.909648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.909735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.909908 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.909989 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.910164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.910254 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.910397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] federation.caching = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.910484 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.910631 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.910711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.910874 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.910954 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.911117 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.911204 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.911345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.911425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.911589 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.911669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.911825 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.911912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.912055 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.912133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.912296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.912380 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.912532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.912618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.912768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.912846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.913009 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.913093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.913238 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity.caching = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.913325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.913485 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.913564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.913728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.913825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.913986 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.914077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.914220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.914299 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.914463 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.914541 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.914691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.914778 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.914927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.915317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.915317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.915451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.915451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.915593 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.915667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.915738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.915833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.915911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.916060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.916147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.916300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.916385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.916531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.916616 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.916795 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.916889 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.917014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.917093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.917255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.917333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.917502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.917588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.917744 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.917840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.918014 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.918093 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.918243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.918324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.918477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.918564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.918704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.918783 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.918936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.919023 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.919171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.919255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.919403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.919481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.919643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.919724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.919866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.919950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.920112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.920196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.920350 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.920429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.920580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.920659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.920822 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.920901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.921069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.921177 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.921308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.921973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.922099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.922211 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.922350 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.922431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.922584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.922663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.922824 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.922903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.923057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.923138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.923297 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.923377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.923536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.923655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.923783 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.923898 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.924012 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.924126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.924239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.924352 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.924473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.924588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.924802 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.924919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.925033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.925145 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.925267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.925382 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.925496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.925608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.925721 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.925853 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.925982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.927946 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.928077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.928195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.928309 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.928423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.928544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.928659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.928772 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.928885 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.928999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.929119 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.929243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.929358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.930350 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.930471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.930586 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.930699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.930827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.930944 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.931066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.931181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.931295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.931408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.931530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.931645 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.931758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.931870 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.931983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.932097 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.932220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.932335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.934881 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.935211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.935211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.935454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.935581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.935698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.935814 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.935927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.936041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.936160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.936283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.936399 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.936513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.936626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.936740 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.936853 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.936975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.937692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.937827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.937942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.938071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.938185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.938375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.938494 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.938607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.938719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.938832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.941940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.942154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.942284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.942697 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.942813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.942927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.943041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.943163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.943279 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.943392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.943512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.943629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.943751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.943866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.943980 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.944092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.944206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] resource.caching = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.944319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.944440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.944556 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.944556 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.944770 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.944883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.944883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.945103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.945218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.945218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.945431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.945551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.945666 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.945789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.945926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.946040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.946164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.946280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.946393 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.946514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.946630 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.946743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] role.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.946858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] role.caching = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.946971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] role.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.947084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] role.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.947205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.947320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.947433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.947546 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.947659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.947772 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.947895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.947895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.948109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.948222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.948334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.948447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.948568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.948568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.948782 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.948896 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.949008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.949008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.949227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.949227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.949445 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.949559 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.949671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.949784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.949931 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.950053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.950170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.950283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.950395 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.950508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.950629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.950744 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.950857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.950971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.951086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.951208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.951322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.951436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.951549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.951662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.951775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.951896 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.952012 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.952126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.952239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.952352 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.952464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.952615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.952737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.952852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.952965 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.953080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.953194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.953316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.953432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.953546 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.953659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.953773 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.953913 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.954040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.954157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.954271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.954385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.954504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.954619 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.954741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.954856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.954969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] token.caching = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.955082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.955196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.955308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.955430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.955545 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.955658 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.955771 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.955885 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.955998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.956123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.956239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.956352 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.956466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.956579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.956691 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.956813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.956928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.957040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.957153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.957266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.957379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.957501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.957616 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.957729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.957857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.957972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.958085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.958210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.958327 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.958450 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.958582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.958707 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.958822 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.958936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.959049 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.connection = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.959162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.connection = **** {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.959287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.959404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.959518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.959632 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.959745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.959858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.960050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.960168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.960282 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.960396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.960510 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.960623 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.960745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.960860 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.960975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.961088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.961201 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.961315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.961438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.961554 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.961668 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.961782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.961943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.962057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.962182 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.962299 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.962412 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.962526 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.962640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.962761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.962876 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.962988 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.963130 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.963247 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.963360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.963481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 19:32:41.963598 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93427) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 14 19:32:41.963712 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93426) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 14 19:32:41.984875 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:41.985101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:41.985291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:41.985390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:41.985529 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:41.985753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:41.993686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:41.994275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.022103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.022334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.023210 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.023444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.076156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.076829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.124659 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.124891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.126266 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.126506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.280635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.281414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.281635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.281877 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.281964 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.282092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.282197 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.282302 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.282385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.282576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.282793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.283010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.305111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.305348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.305570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.305652 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.305790 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.305899 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.306025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.306150 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.306232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.306390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.306486 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.306619 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.306840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.307055 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.318302 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.319067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.331524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.331705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.331878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.331974 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.332116 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.332201 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.332331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.332413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.332540 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.332632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.332750 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93426) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.332834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.332965 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.333185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93427) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 19:32:42.349252 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.349456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 19:32:42.349542 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.349674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 19:32:42.349830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.350022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 19:32:42.350117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.350236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 19:32:42.350318 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.350514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.350598 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.350777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.350867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.351030 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.351110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.351286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.360620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.360899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.361120 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.361201 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.361403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.361484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.361679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.361991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.368333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.368963 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.371228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.371513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.371796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.371879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.372077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.372160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.372360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.372451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.372623 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.372706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.372893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.372976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.373165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.373256 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.373432 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.373514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.373701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.373784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.373991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.374075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.374293 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.375151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.397375 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.398425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.398843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.399124 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.399931 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.400228 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.403738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.404022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.404300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.404989 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.405279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.405566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.410630 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.410919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.412064 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.412352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.416469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.416770 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.417055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.417333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.417605 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.417900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.418171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.418265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.418436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.418563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.418858 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.419148 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.419432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.419715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.419993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.420267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.433914 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.434191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.435720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.436002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.438576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.440349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.442060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.442353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.443836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.444132 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.446569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.448368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.449626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.451638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.452890 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.453187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.453472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.453757 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.454058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.454338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.455003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.455308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.455600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.455888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.456178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.456457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.470230 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.470511 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.470779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.471039 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.472695 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.472983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.473257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.473525 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.477703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.479225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.480431 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.480709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.480981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.481995 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.483565 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.483744 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.483900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.484110 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.484395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.484681 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.485067 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.486700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.486990 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.487278 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.487564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.487967 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.496142 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.496422 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.496698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.496973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.497348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.497642 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.497943 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.499138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.499423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.500225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.500509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.500891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.501171 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.501450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.513649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.516706 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.516908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.519669 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.519896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.521152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.522635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.522817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.522911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.523290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.524326 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.525781 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.526088 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.526486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.529636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.531670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.533381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.534555 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.534905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.534990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.535219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.535498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.537779 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.538205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.538507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.538792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.543261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.543547 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.543824 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.546752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.547043 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.547331 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.550735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93426) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.551014 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.551292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.551563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.551842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.552112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.552381 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.552660 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.552926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.553200 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.553507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.553820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.554147 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.554231 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93427) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 19:32:42.554435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93426) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.554517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.554756 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.555038 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.555318 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.555612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.555902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.556181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.556453 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.556731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.557014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.557311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.557581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.557904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93427) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 19:32:42.589213 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c7956619c0 pid: 93426 (default app) Sep 14 19:32:42.592417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ce8ff-2d86-4e0e-b862-298f7d4f75f5 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:42.592703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ce8ff-2d86-4e0e-b862-298f7d4f75f5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:42.592973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ce8ff-2d86-4e0e-b862-298f7d4f75f5 None None] PATH_INFO: `/v3` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:42.593128 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c7956619c0 pid: 93427 (default app) Sep 14 19:32:42.597153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1/1] 10.222.0.42 () {50 vars in 791 bytes} [Wed Sep 14 19:32:42 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:32:50.414890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035cb92-1ed0-409e-b110-85e7aa247f7e None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:50.415464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035cb92-1ed0-409e-b110-85e7aa247f7e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:50.415464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035cb92-1ed0-409e-b110-85e7aa247f7e None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:50.419647 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1/2] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:32:50 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:32:50.425336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63ee7e-9a5c-46e4-a897-0134b7a80d59 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:50.425904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63ee7e-9a5c-46e4-a897-0134b7a80d59 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:50.425904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e63ee7e-9a5c-46e4-a897-0134b7a80d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:50.497059 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: INFO dbcounter [None req-9e63ee7e-9a5c-46e4-a897-0134b7a80d59 None None] Registered counter for database keystone Sep 14 19:32:50.517659 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG oslo_db.sqlalchemy.engines [None req-9e63ee7e-9a5c-46e4-a897-0134b7a80d59 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93426) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 14 19:32:50.523842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writer thread running {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 14 19:32:50.703088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG passlib.handlers.bcrypt [None req-9e63ee7e-9a5c-46e4-a897-0134b7a80d59 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93426) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 14 19:32:50.703400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG passlib.handlers.bcrypt [None req-9e63ee7e-9a5c-46e4-a897-0134b7a80d59 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93426) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 14 19:32:50.761157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-9e63ee7e-9a5c-46e4-a897-0134b7a80d59 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:32:50.762820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-9e63ee7e-9a5c-46e4-a897-0134b7a80d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:32:50.889125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 2/3] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:32:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 14 19:32:51.364067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035cb92-1ed0-409e-b110-85e7aa247f7e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:32:51.439479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: INFO dbcounter [None req-d035cb92-1ed0-409e-b110-85e7aa247f7e None None] Registered counter for database keystone Sep 14 19:32:51.459017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG oslo_db.sqlalchemy.engines [None req-d035cb92-1ed0-409e-b110-85e7aa247f7e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93427) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 14 19:32:51.465116 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writer thread running {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 14 19:32:51.581027 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035cb92-1ed0-409e-b110-85e7aa247f7e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:32:51.585364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c250b4d-35e9-45a6-858b-445529eb55d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:32:51.586034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c250b4d-35e9-45a6-858b-445529eb55d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:51.586319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c250b4d-35e9-45a6-858b-445529eb55d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:51.586588 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c250b4d-35e9-45a6-858b-445529eb55d5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:51.594845 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c250b4d-35e9-45a6-858b-445529eb55d5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:32:51.646482 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: WARNING keystone.common.rbac_enforcer.enforcer [None req-7c250b4d-35e9-45a6-858b-445529eb55d5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 14 19:32:51.646773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c250b4d-35e9-45a6-858b-445529eb55d5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:32:51.655885 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 2/4] 10.222.0.42 () {58 vars in 1324 bytes} [Wed Sep 14 19:32:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 19:32:51.661878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63ee7e-9a5c-46e4-a897-0134b7a80d59 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:32:51.678767 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e63ee7e-9a5c-46e4-a897-0134b7a80d59 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:32:51.683160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb71e942-aba1-4a19-b941-aaaec73fdce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:32:51.684257 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71e942-aba1-4a19-b941-aaaec73fdce5 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:51.684536 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71e942-aba1-4a19-b941-aaaec73fdce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:51.684814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb71e942-aba1-4a19-b941-aaaec73fdce5 admin admin] PATH_INFO: `/v3/projects/2a8f053b53ae4aff97a668669e77d899` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:51.686416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb71e942-aba1-4a19-b941-aaaec73fdce5 admin admin] RBAC: Authorizing `identity:get_project(project_id=2a8f053b53ae4aff97a668669e77d899)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:32:51.740141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cb71e942-aba1-4a19-b941-aaaec73fdce5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 14 19:32:51.740481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb71e942-aba1-4a19-b941-aaaec73fdce5 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:32:51.744954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 3/5] 10.222.0.42 () {56 vars in 1207 bytes} [Wed Sep 14 19:32:51 2022] GET /identity/v3/projects/2a8f053b53ae4aff97a668669e77d899 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:32:53.848344 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd0d8-48e4-421b-b5e6-74c276d50d1f None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:53.848870 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd0d8-48e4-421b-b5e6-74c276d50d1f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:53.848870 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646cd0d8-48e4-421b-b5e6-74c276d50d1f None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:53.852911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 3/6] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:32:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:32:53.858246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a21278-c716-48bb-9241-b3f9ad5b8685 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:53.858682 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a21278-c716-48bb-9241-b3f9ad5b8685 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:53.858776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a21278-c716-48bb-9241-b3f9ad5b8685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:53.878399 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-97a21278-c716-48bb-9241-b3f9ad5b8685 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:32:53.879924 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-97a21278-c716-48bb-9241-b3f9ad5b8685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:32:53.887667 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 4/7] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:32:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 14 19:32:54.150984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cd0d8-48e4-421b-b5e6-74c276d50d1f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:32:54.163337 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646cd0d8-48e4-421b-b5e6-74c276d50d1f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:32:54.167467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b30b09-36fa-4dda-8f5b-5b51b7122216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:32:54.168053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b30b09-36fa-4dda-8f5b-5b51b7122216 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:54.168325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b30b09-36fa-4dda-8f5b-5b51b7122216 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:54.168588 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b30b09-36fa-4dda-8f5b-5b51b7122216 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:54.175998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b30b09-36fa-4dda-8f5b-5b51b7122216 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:32:54.177751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b30b09-36fa-4dda-8f5b-5b51b7122216 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:32:54.186329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 4/8] 10.222.0.42 () {58 vars in 1324 bytes} [Wed Sep 14 19:32:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 19:32:54.192234 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a21278-c716-48bb-9241-b3f9ad5b8685 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:32:54.201091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a21278-c716-48bb-9241-b3f9ad5b8685 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:32:54.205945 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e790d3-5da4-4527-adcd-fed6708cc553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:32:54.206557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e790d3-5da4-4527-adcd-fed6708cc553 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:54.206833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e790d3-5da4-4527-adcd-fed6708cc553 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:54.207107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e790d3-5da4-4527-adcd-fed6708cc553 admin admin] PATH_INFO: `/v3/users/e8f45104d2d540658c4bbc32613a9317` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:54.208404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e790d3-5da4-4527-adcd-fed6708cc553 admin admin] RBAC: Authorizing `identity:get_user(user_id=e8f45104d2d540658c4bbc32613a9317)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:32:54.210241 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e790d3-5da4-4527-adcd-fed6708cc553 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:32:54.211726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 5/9] 10.222.0.42 () {56 vars in 1198 bytes} [Wed Sep 14 19:32:54 2022] GET /identity/v3/users/e8f45104d2d540658c4bbc32613a9317 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:32:56.413588 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefe465-b50b-4569-9a12-36b82041e55c None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:56.414164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefe465-b50b-4569-9a12-36b82041e55c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:56.414164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfefe465-b50b-4569-9a12-36b82041e55c None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:56.415515 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 5/10] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:32:56.420780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39e8c7-3fe6-4157-b49d-5851adc94a59 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:56.421730 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39e8c7-3fe6-4157-b49d-5851adc94a59 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:56.421943 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39e8c7-3fe6-4157-b49d-5851adc94a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:56.440250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-eb39e8c7-3fe6-4157-b49d-5851adc94a59 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:32:56.441441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-eb39e8c7-3fe6-4157-b49d-5851adc94a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:32:56.444235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59985c-2ade-4ee6-ba60-8a80c06e9e4f None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:56.444522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59985c-2ade-4ee6-ba60-8a80c06e9e4f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:56.444795 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59985c-2ade-4ee6-ba60-8a80c06e9e4f None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:56.446135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 6/11] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:32:56.451311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24c36b-8371-41ae-9750-53324b37aa67 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:56.451593 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24c36b-8371-41ae-9750-53324b37aa67 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:56.451864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24c36b-8371-41ae-9750-53324b37aa67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:56.493606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG passlib.handlers.bcrypt [None req-2d24c36b-8371-41ae-9750-53324b37aa67 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93427) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 14 19:32:56.494015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG passlib.handlers.bcrypt [None req-2d24c36b-8371-41ae-9750-53324b37aa67 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93427) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 14 19:32:56.522017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 6/12] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:32:56.542638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2d24c36b-8371-41ae-9750-53324b37aa67 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:32:56.544239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2d24c36b-8371-41ae-9750-53324b37aa67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:32:56.641079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 7/13] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:32:56.797053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39e8c7-3fe6-4157-b49d-5851adc94a59 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:32:56.820436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:32:56.899091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39e8c7-3fe6-4157-b49d-5851adc94a59 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:32:56.904726 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24c36b-8371-41ae-9750-53324b37aa67 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:32:56.932170 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a60be-51ed-4f3a-8291-138b029fa833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:32:56.932807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a60be-51ed-4f3a-8291-138b029fa833 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:56.933079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a60be-51ed-4f3a-8291-138b029fa833 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:56.933357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a60be-51ed-4f3a-8291-138b029fa833 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:56.944720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a60be-51ed-4f3a-8291-138b029fa833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-314a60be-51ed-4f3a-8291-138b029fa833 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:32:56.953637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:32:56.954575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:32:56.955417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:32:56.955417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:32:56.955417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:32:56.955417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:32:56.955417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:32:56.955417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:32:56.955417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:32:56.955417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:32:56.955417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:32:56.955417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:32:56.955417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:32:56.955417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:32:56.955417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 7/14] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:32:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:32:56.960483 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a60be-51ed-4f3a-8291-138b029fa833 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:32:57.010506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24c36b-8371-41ae-9750-53324b37aa67 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:32:57.012839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:32:57.041595 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:32:57.047467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c69c70-aabb-4b0c-850b-d541630ea69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:32:57.048263 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c69c70-aabb-4b0c-850b-d541630ea69d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:57.048585 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c69c70-aabb-4b0c-850b-d541630ea69d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:57.048856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c69c70-aabb-4b0c-850b-d541630ea69d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:57.059011 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a60be-51ed-4f3a-8291-138b029fa833 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:32:57.091833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d1a52-79ef-47f3-9244-265dbba92acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:32:57.092429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d1a52-79ef-47f3-9244-265dbba92acc None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:57.092702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d1a52-79ef-47f3-9244-265dbba92acc None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:57.092965 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d1a52-79ef-47f3-9244-265dbba92acc None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:57.094186 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d1a52-79ef-47f3-9244-265dbba92acc None admin] RBAC: Adding query filter params (name=admin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:32:57.094482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d1a52-79ef-47f3-9244-265dbba92acc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:32:57.095850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d1a52-79ef-47f3-9244-265dbba92acc None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:32:57.107927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 8/15] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:32:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:32:57.113556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d1a52-79ef-47f3-9244-265dbba92acc None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:32:57.115893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c69c70-aabb-4b0c-850b-d541630ea69d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:32:57.117524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c69c70-aabb-4b0c-850b-d541630ea69d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:32:57.209882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d1a52-79ef-47f3-9244-265dbba92acc None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:32:57.214094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 8/16] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:32:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 19:32:57.220110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c69c70-aabb-4b0c-850b-d541630ea69d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:32:57.222767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:32:57.243499 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cf25a-ae3b-44cc-9aee-a8fab0b76518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:32:57.244122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cf25a-ae3b-44cc-9aee-a8fab0b76518 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:57.244400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cf25a-ae3b-44cc-9aee-a8fab0b76518 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:57.244668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cf25a-ae3b-44cc-9aee-a8fab0b76518 None admin] PATH_INFO: `/v3/users/e8f45104d2d540658c4bbc32613a9317` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:57.245825 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00cf25a-ae3b-44cc-9aee-a8fab0b76518 None admin] RBAC: Authorizing `identity:get_user(user_id=e8f45104d2d540658c4bbc32613a9317)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:32:57.247274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00cf25a-ae3b-44cc-9aee-a8fab0b76518 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:32:57.249091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 9/17] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:32:57 2022] GET /identity/v3/users/e8f45104d2d540658c4bbc32613a9317 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:32:57.254091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cf25a-ae3b-44cc-9aee-a8fab0b76518 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:32:57.267700 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:32:57.317328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c69c70-aabb-4b0c-850b-d541630ea69d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:32:57.350737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cf25a-ae3b-44cc-9aee-a8fab0b76518 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:32:57.351815 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6334af6-6004-43f2-8bd0-539db2c840b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:32:57.352738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334af6-6004-43f2-8bd0-539db2c840b9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:57.353014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334af6-6004-43f2-8bd0-539db2c840b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:57.353285 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6334af6-6004-43f2-8bd0-539db2c840b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:57.372123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6334af6-6004-43f2-8bd0-539db2c840b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:32:57.373636 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6334af6-6004-43f2-8bd0-539db2c840b9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:32:57.384156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d11b-1ca0-48c1-a3e0-9711b63ff60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:32:57.384784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d11b-1ca0-48c1-a3e0-9711b63ff60d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:57.385062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d11b-1ca0-48c1-a3e0-9711b63ff60d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:57.385333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba5d11b-1ca0-48c1-a3e0-9711b63ff60d None admin] PATH_INFO: `/v3/domains/default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:57.386450 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5d11b-1ca0-48c1-a3e0-9711b63ff60d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:32:57.387882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba5d11b-1ca0-48c1-a3e0-9711b63ff60d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:32:57.389445 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 10/18] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:32:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-b6334af6-6004-43f2-8bd0-539db2c840b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:32:57.391872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:32:57.392730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:32:57.393591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:32:57.393591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:32:57.393591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:32:57.393591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 9/19] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:32:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:32:57.395382 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d11b-1ca0-48c1-a3e0-9711b63ff60d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:32:57.398436 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6334af6-6004-43f2-8bd0-539db2c840b9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:32:57.454689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:32:57.476138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:32:57.494158 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba5d11b-1ca0-48c1-a3e0-9711b63ff60d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:32:57.497973 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6334af6-6004-43f2-8bd0-539db2c840b9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:32:57.529311 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8f094-e0de-45e1-922b-1cd5ddc6e15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:32:57.529944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8f094-e0de-45e1-922b-1cd5ddc6e15b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:57.530221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8f094-e0de-45e1-922b-1cd5ddc6e15b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:57.530500 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8f094-e0de-45e1-922b-1cd5ddc6e15b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:57.531229 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8f094-e0de-45e1-922b-1cd5ddc6e15b None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.domain.id=default, user.id=e8f45104d2d540658c4bbc32613a9317) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:32:57.531532 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8f094-e0de-45e1-922b-1cd5ddc6e15b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:32:57.531754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5fe7a-7ca4-4fec-b870-c34cf3af1a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:32:57.532357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5fe7a-7ca4-4fec-b870-c34cf3af1a5c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:57.532635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5fe7a-7ca4-4fec-b870-c34cf3af1a5c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:57.532905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5fe7a-7ca4-4fec-b870-c34cf3af1a5c None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:57.533006 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c8f094-e0de-45e1-922b-1cd5ddc6e15b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:32:57.533535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5fe7a-7ca4-4fec-b870-c34cf3af1a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:32:57.533858 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5fe7a-7ca4-4fec-b870-c34cf3af1a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:32:57.535224 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5fe7a-7ca4-4fec-b870-c34cf3af1a5c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:32:57.539715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 11/20] 10.222.0.42 () {56 vars in 1322 bytes} [Wed Sep 14 19:32:57 2022] GET /identity/v3/role_assignments?user.id=e8f45104d2d540658c4bbc32613a9317&scope.domain.id=default&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 233 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:32:57.553204 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 10/21] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:32:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:32:59.965593 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736af463-3cf8-4b2b-a8ee-08917c95f26c None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:59.966604 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736af463-3cf8-4b2b-a8ee-08917c95f26c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:59.967136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736af463-3cf8-4b2b-a8ee-08917c95f26c None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:32:59.968786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 12/22] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:32:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:32:59.974857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7791a85-961b-4c1c-b5f4-fcfe6af82be1 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:32:59.975819 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7791a85-961b-4c1c-b5f4-fcfe6af82be1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:32:59.976297 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7791a85-961b-4c1c-b5f4-fcfe6af82be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:00.001141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a7791a85-961b-4c1c-b5f4-fcfe6af82be1 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:00.002722 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a7791a85-961b-4c1c-b5f4-fcfe6af82be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:00.021371 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb8cba-699e-48fa-83be-9a6d5c30a395 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:00.021985 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb8cba-699e-48fa-83be-9a6d5c30a395 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:00.022473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb8cba-699e-48fa-83be-9a6d5c30a395 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:00.024050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 13/23] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:33:00.031290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c312b9-7a60-476d-966d-4fa4ce970507 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:00.031951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c312b9-7a60-476d-966d-4fa4ce970507 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:00.032425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c312b9-7a60-476d-966d-4fa4ce970507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:00.061380 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-a0c312b9-7a60-476d-966d-4fa4ce970507 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:00.064178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a0c312b9-7a60-476d-966d-4fa4ce970507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:00.151694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 11/24] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:32:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:00.179116 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28001cc-b656-4561-a32a-89cf56efce80 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:00.179737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28001cc-b656-4561-a32a-89cf56efce80 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:00.180210 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28001cc-b656-4561-a32a-89cf56efce80 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:00.181736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 12/25] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:33:00.194950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2524e2-5177-4d67-87a9-a7a255eb7c09 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:00.195564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2524e2-5177-4d67-87a9-a7a255eb7c09 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:00.196063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2524e2-5177-4d67-87a9-a7a255eb7c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:00.236439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bc2524e2-5177-4d67-87a9-a7a255eb7c09 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:00.238191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bc2524e2-5177-4d67-87a9-a7a255eb7c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:00.255986 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 14/26] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:00.305136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:00.347901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 13/27] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 14 19:33:00.475539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c312b9-7a60-476d-966d-4fa4ce970507 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:00.570424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:00.596459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c312b9-7a60-476d-966d-4fa4ce970507 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:00.632139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280e8bf-4288-4fc1-9806-69b2eec80568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:00.633127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280e8bf-4288-4fc1-9806-69b2eec80568 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:00.633600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280e8bf-4288-4fc1-9806-69b2eec80568 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:00.634082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280e8bf-4288-4fc1-9806-69b2eec80568 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:00.645195 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7280e8bf-4288-4fc1-9806-69b2eec80568 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-7280e8bf-4288-4fc1-9806-69b2eec80568 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:00.650317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:00.651155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:00.651917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:00.651917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:00.651917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:00.651917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:00.651917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:00.651917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:00.651917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:00.651917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:00.651917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:00.651917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:00.651917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:00.651917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:00.654164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 15/28] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:33:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:00.660169 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2524e2-5177-4d67-87a9-a7a255eb7c09 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:00.750743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280e8bf-4288-4fc1-9806-69b2eec80568 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:00.826496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2524e2-5177-4d67-87a9-a7a255eb7c09 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:00.867473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad368d5c-f67f-426f-b290-d09091eacb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:00.868581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad368d5c-f67f-426f-b290-d09091eacb29 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:00.868995 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad368d5c-f67f-426f-b290-d09091eacb29 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:00.869469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad368d5c-f67f-426f-b290-d09091eacb29 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:00.870382 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad368d5c-f67f-426f-b290-d09091eacb29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:00.870825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad368d5c-f67f-426f-b290-d09091eacb29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:00.872357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad368d5c-f67f-426f-b290-d09091eacb29 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:00.886828 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 14/29] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:00.887941 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad368d5c-f67f-426f-b290-d09091eacb29 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:00.923033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280e8bf-4288-4fc1-9806-69b2eec80568 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:00.961788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb1280-39d1-4e4a-b50a-ae1a14d81ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:00.962719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb1280-39d1-4e4a-b50a-ae1a14d81ab6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:00.963165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb1280-39d1-4e4a-b50a-ae1a14d81ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:00.963572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb1280-39d1-4e4a-b50a-ae1a14d81ab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:00.966573 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:00.970104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:00.979056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb1280-39d1-4e4a-b50a-ae1a14d81ab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:00.980807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb1280-39d1-4e4a-b50a-ae1a14d81ab6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-23eb1280-39d1-4e4a-b50a-ae1a14d81ab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:33:00.997622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:00.998551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:33:00.999884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:33:00.999884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:33:00.999884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:01.004289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 16/30] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:01.005274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb1280-39d1-4e4a-b50a-ae1a14d81ab6 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:01.039473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad368d5c-f67f-426f-b290-d09091eacb29 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:01.075355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959794f6-edb7-42e2-bfee-5899d406a40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:01.076180 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959794f6-edb7-42e2-bfee-5899d406a40f None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:01.076687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959794f6-edb7-42e2-bfee-5899d406a40f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:01.077110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959794f6-edb7-42e2-bfee-5899d406a40f None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:01.078099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959794f6-edb7-42e2-bfee-5899d406a40f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:01.079612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959794f6-edb7-42e2-bfee-5899d406a40f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:01.113666 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb1280-39d1-4e4a-b50a-ae1a14d81ab6 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:01.148408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd696d5-3f1d-4c13-aff3-51f176494b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:01.149030 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd696d5-3f1d-4c13-aff3-51f176494b24 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:01.149309 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd696d5-3f1d-4c13-aff3-51f176494b24 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:01.149579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd696d5-3f1d-4c13-aff3-51f176494b24 None admin] PATH_INFO: `/v3/users/e8f45104d2d540658c4bbc32613a9317` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:01.150741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd696d5-3f1d-4c13-aff3-51f176494b24 None admin] RBAC: Authorizing `identity:get_user(user_id=e8f45104d2d540658c4bbc32613a9317)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:01.152165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd696d5-3f1d-4c13-aff3-51f176494b24 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:01.158861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 17/31] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:01 2022] GET /identity/v3/users/e8f45104d2d540658c4bbc32613a9317 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:01.158861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d6201-dc66-4c50-a96b-5901c73c1af1 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:01.158861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d6201-dc66-4c50-a96b-5901c73c1af1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:01.158861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356d6201-dc66-4c50-a96b-5901c73c1af1 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:01.158861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 18/32] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:01.158861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d6201-dc66-4c50-a96b-5901c73c1af1 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:01.233246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:01.283758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356d6201-dc66-4c50-a96b-5901c73c1af1 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:01.289227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 15/33] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Sep 14 19:33:00 2022] POST /identity/v3/roles => generated 219 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:33:01.290385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959794f6-edb7-42e2-bfee-5899d406a40f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:01.334473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a9859-eb28-4db3-9706-9c3f96118108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:01.336246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a9859-eb28-4db3-9706-9c3f96118108 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:01.336735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a9859-eb28-4db3-9706-9c3f96118108 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:01.337197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a9859-eb28-4db3-9706-9c3f96118108 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:01.338145 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a9859-eb28-4db3-9706-9c3f96118108 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:01.339170 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a9859-eb28-4db3-9706-9c3f96118108 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:01.341580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a9859-eb28-4db3-9706-9c3f96118108 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:01.368515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 19/34] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:01.370123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efab7e-eef8-4224-a473-9af4ab21e151 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:01.370695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efab7e-eef8-4224-a473-9af4ab21e151 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:01.371157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67efab7e-eef8-4224-a473-9af4ab21e151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:01.397142 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-67efab7e-eef8-4224-a473-9af4ab21e151 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:01.399923 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-67efab7e-eef8-4224-a473-9af4ab21e151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:01.404230 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:01.414731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959794f6-edb7-42e2-bfee-5899d406a40f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:01.466115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b7a17-3244-43fd-8a7a-da1eb213812b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:01.467477 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b7a17-3244-43fd-8a7a-da1eb213812b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:01.467970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b7a17-3244-43fd-8a7a-da1eb213812b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:01.468763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b7a17-3244-43fd-8a7a-da1eb213812b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:01.471546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b7a17-3244-43fd-8a7a-da1eb213812b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:01.474236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7b7a17-3244-43fd-8a7a-da1eb213812b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:01.476824 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 16/35] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:01.478910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b7a17-3244-43fd-8a7a-da1eb213812b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:01.486669 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 20/36] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:01.487640 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efab7e-eef8-4224-a473-9af4ab21e151 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:01.583547 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:01.660195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b7a17-3244-43fd-8a7a-da1eb213812b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:01.692191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67efab7e-eef8-4224-a473-9af4ab21e151 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:01.729208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c567db-6891-4b06-ba0c-43a7739051f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:01.731054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c567db-6891-4b06-ba0c-43a7739051f5 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:01.731810 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c567db-6891-4b06-ba0c-43a7739051f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:01.733186 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c567db-6891-4b06-ba0c-43a7739051f5 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:01.735772 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c567db-6891-4b06-ba0c-43a7739051f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:01.737465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c567db-6891-4b06-ba0c-43a7739051f5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:01.774072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194fdf2-a1da-4cc9-871c-711558b79d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:01.775032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194fdf2-a1da-4cc9-871c-711558b79d14 None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:01.775747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194fdf2-a1da-4cc9-871c-711558b79d14 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:01.776223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194fdf2-a1da-4cc9-871c-711558b79d14 None admin] PATH_INFO: `/v3/domains/default/users/e8f45104d2d540658c4bbc32613a9317/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:01.779041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194fdf2-a1da-4cc9-871c-711558b79d14 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e8f45104d2d540658c4bbc32613a9317, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:01.781388 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194fdf2-a1da-4cc9-871c-711558b79d14 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:01.880083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 17/37] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 19:33:01 2022] POST /identity/v3/projects => generated 301 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:33:01.881420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a7587-b342-4285-a4aa-7657ef841793 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:01.881709 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a7587-b342-4285-a4aa-7657ef841793 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:01.882352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a7587-b342-4285-a4aa-7657ef841793 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:01.883518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 18/38] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:01.886720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a7587-b342-4285-a4aa-7657ef841793 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:01.910108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:02.008618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a7587-b342-4285-a4aa-7657ef841793 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:02.039506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 21/39] 10.222.0.42 () {58 vars in 1361 bytes} [Wed Sep 14 19:33:01 2022] PUT /identity/v3/domains/default/users/e8f45104d2d540658c4bbc32613a9317/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:02.042366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e259a7-1a3a-493e-8bc9-e9148f88b725 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:02.042888 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e259a7-1a3a-493e-8bc9-e9148f88b725 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:02.043469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e259a7-1a3a-493e-8bc9-e9148f88b725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:02.048372 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad83b3-8595-4a33-a173-6ca1158c98bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:02.049049 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad83b3-8595-4a33-a173-6ca1158c98bb None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:02.049340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad83b3-8595-4a33-a173-6ca1158c98bb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:02.049617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad83b3-8595-4a33-a173-6ca1158c98bb None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:02.050346 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad83b3-8595-4a33-a173-6ca1158c98bb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:02.051792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad83b3-8595-4a33-a173-6ca1158c98bb None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:02.068719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-d9e259a7-1a3a-493e-8bc9-e9148f88b725 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:02.070946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d9e259a7-1a3a-493e-8bc9-e9148f88b725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:02.126006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 19/40] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Sep 14 19:33:01 2022] POST /identity/v3/roles => generated 225 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:33:02.127809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18929240-258f-44c4-b43a-981c8b2cbca1 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:02.128296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18929240-258f-44c4-b43a-981c8b2cbca1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:02.128790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18929240-258f-44c4-b43a-981c8b2cbca1 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:02.130368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 20/41] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:02.140731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a9bc5-f6f1-458d-a00b-6ca92cc03aa5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:02.141063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a9bc5-f6f1-458d-a00b-6ca92cc03aa5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:02.141283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8a9bc5-f6f1-458d-a00b-6ca92cc03aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:02.160953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 22/42] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:02.162282 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8b8a9bc5-f6f1-458d-a00b-6ca92cc03aa5 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:02.163932 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8b8a9bc5-f6f1-458d-a00b-6ca92cc03aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:02.251801 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 21/43] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:02.448842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e259a7-1a3a-493e-8bc9-e9148f88b725 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:02.483108 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:02.517710 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a9bc5-f6f1-458d-a00b-6ca92cc03aa5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:02.528911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:02.622941 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8a9bc5-f6f1-458d-a00b-6ca92cc03aa5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:02.657562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35c8c6-5f03-4529-b348-eb7e12eebff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:02.658220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35c8c6-5f03-4529-b348-eb7e12eebff3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:02.658499 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35c8c6-5f03-4529-b348-eb7e12eebff3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:02.658767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b35c8c6-5f03-4529-b348-eb7e12eebff3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:02.659851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35c8c6-5f03-4529-b348-eb7e12eebff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:02.660854 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e259a7-1a3a-493e-8bc9-e9148f88b725 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:02.661280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b35c8c6-5f03-4529-b348-eb7e12eebff3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:02.663124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 22/44] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:02.668289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35c8c6-5f03-4529-b348-eb7e12eebff3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:02.698069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5426aa2-a192-484d-9305-f423687f5e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:02.698686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5426aa2-a192-484d-9305-f423687f5e7c None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:02.699016 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5426aa2-a192-484d-9305-f423687f5e7c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:02.699238 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5426aa2-a192-484d-9305-f423687f5e7c None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:02.699904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5426aa2-a192-484d-9305-f423687f5e7c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:02.701481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5426aa2-a192-484d-9305-f423687f5e7c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:02.748982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:02.770607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b35c8c6-5f03-4529-b348-eb7e12eebff3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:02.800406 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 23/45] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Sep 14 19:33:02 2022] POST /identity/v3/roles => generated 223 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:33:02.804518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5556b9-db57-4548-b818-be0633111a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:02.805143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5556b9-db57-4548-b818-be0633111a72 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:02.805422 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5556b9-db57-4548-b818-be0633111a72 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:02.805693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5556b9-db57-4548-b818-be0633111a72 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:02.806436 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5556b9-db57-4548-b818-be0633111a72 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:02.808571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5556b9-db57-4548-b818-be0633111a72 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:02.923087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 23/46] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 19:33:02 2022] POST /identity/v3/projects => generated 312 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:33:04.262699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf08cb-c9f5-4f88-962d-995442bdd2dd None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:04.263188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf08cb-c9f5-4f88-962d-995442bdd2dd None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:04.263188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbf08cb-c9f5-4f88-962d-995442bdd2dd None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:04.264515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 24/47] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:04.270556 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a1e19-d6d1-4f76-a20c-3ab62153e3fe None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:04.270976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a1e19-d6d1-4f76-a20c-3ab62153e3fe None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:04.271114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a1e19-d6d1-4f76-a20c-3ab62153e3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:04.288886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-885a1e19-d6d1-4f76-a20c-3ab62153e3fe None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:04.290092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-885a1e19-d6d1-4f76-a20c-3ab62153e3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:04.353734 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 24/48] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:04.619540 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf08cb-c9f5-4f88-962d-995442bdd2dd None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:04.712162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:04.720189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbf08cb-c9f5-4f88-962d-995442bdd2dd None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:04.753468 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fd04a-d06c-46e4-94e4-53b17f2811e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:04.754138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fd04a-d06c-46e4-94e4-53b17f2811e2 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:04.754618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fd04a-d06c-46e4-94e4-53b17f2811e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:04.755061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fd04a-d06c-46e4-94e4-53b17f2811e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:04.766849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fd04a-d06c-46e4-94e4-53b17f2811e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-d88fd04a-d06c-46e4-94e4-53b17f2811e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:04.771835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:04.772698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:04.773453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:04.773453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:04.773453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:04.773453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:04.773453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:04.773453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:04.773453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:04.773453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:04.773453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:04.773453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:04.773453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:04.773453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:04.773453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 25/49] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:33:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:04.778814 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a1e19-d6d1-4f76-a20c-3ab62153e3fe None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:04.835591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:04.878498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a1e19-d6d1-4f76-a20c-3ab62153e3fe None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:04.912338 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9060a0-0db5-4d6a-8fdc-2d617515cb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:04.912948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9060a0-0db5-4d6a-8fdc-2d617515cb71 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:04.913231 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9060a0-0db5-4d6a-8fdc-2d617515cb71 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:04.913505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9060a0-0db5-4d6a-8fdc-2d617515cb71 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:04.914233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9060a0-0db5-4d6a-8fdc-2d617515cb71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:04.914539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9060a0-0db5-4d6a-8fdc-2d617515cb71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:04.915924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9060a0-0db5-4d6a-8fdc-2d617515cb71 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:04.924131 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 25/50] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:04.930081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fd04a-d06c-46e4-94e4-53b17f2811e2 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:05.032559 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fd04a-d06c-46e4-94e4-53b17f2811e2 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:05.066053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ab839-0ed4-40fd-bf97-707a30d1bf5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:05.066685 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ab839-0ed4-40fd-bf97-707a30d1bf5c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:05.067224 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ab839-0ed4-40fd-bf97-707a30d1bf5c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:05.067501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ab839-0ed4-40fd-bf97-707a30d1bf5c None admin] PATH_INFO: `/v3/users/e8f45104d2d540658c4bbc32613a9317` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:05.068636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ab839-0ed4-40fd-bf97-707a30d1bf5c None admin] RBAC: Authorizing `identity:get_user(user_id=e8f45104d2d540658c4bbc32613a9317)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:05.070123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ab839-0ed4-40fd-bf97-707a30d1bf5c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:05.071715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 26/51] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:04 2022] GET /identity/v3/users/e8f45104d2d540658c4bbc32613a9317 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:05.073006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe92c09-2156-4450-b124-dc4f5456de25 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:05.073329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe92c09-2156-4450-b124-dc4f5456de25 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:05.073612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe92c09-2156-4450-b124-dc4f5456de25 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:05.074976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 26/52] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:33:05.076961 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ab839-0ed4-40fd-bf97-707a30d1bf5c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:05.079911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a9f8b-7a09-412e-a8df-e69590720152 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:05.080199 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a9f8b-7a09-412e-a8df-e69590720152 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:05.080479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585a9f8b-7a09-412e-a8df-e69590720152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:05.098060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-585a9f8b-7a09-412e-a8df-e69590720152 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:05.099261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-585a9f8b-7a09-412e-a8df-e69590720152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:05.101601 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:05.166342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 27/53] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:05.180227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ab839-0ed4-40fd-bf97-707a30d1bf5c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:05.213298 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad5fea-a7b4-4d67-917f-8abfcd6497f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:05.213952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad5fea-a7b4-4d67-917f-8abfcd6497f7 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:05.214236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad5fea-a7b4-4d67-917f-8abfcd6497f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:05.214520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad5fea-a7b4-4d67-917f-8abfcd6497f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:05.215688 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad5fea-a7b4-4d67-917f-8abfcd6497f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:05.217134 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad5fea-a7b4-4d67-917f-8abfcd6497f7 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:05.219415 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 27/54] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:05.225163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a9f8b-7a09-412e-a8df-e69590720152 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:05.375150 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:05.423052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585a9f8b-7a09-412e-a8df-e69590720152 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:05.433531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad5fea-a7b4-4d67-917f-8abfcd6497f7 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:05.458047 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c211a6-f8c2-4e95-be38-b367b4eea216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:05.458659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c211a6-f8c2-4e95-be38-b367b4eea216 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:05.458943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c211a6-f8c2-4e95-be38-b367b4eea216 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:05.459220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c211a6-f8c2-4e95-be38-b367b4eea216 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:05.459941 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c211a6-f8c2-4e95-be38-b367b4eea216 None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.domain.id=default, user.id=e8f45104d2d540658c4bbc32613a9317) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:05.460249 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c211a6-f8c2-4e95-be38-b367b4eea216 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:05.461695 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c211a6-f8c2-4e95-be38-b367b4eea216 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:05.470990 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 28/55] 10.222.0.42 () {56 vars in 1322 bytes} [Wed Sep 14 19:33:05 2022] GET /identity/v3/role_assignments?user.id=e8f45104d2d540658c4bbc32613a9317&scope.domain.id=default&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 530 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:05.528636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:05.539190 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad5fea-a7b4-4d67-917f-8abfcd6497f7 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:05.576725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e08fc9-28c7-49f7-9880-46c82f64a7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:05.577938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e08fc9-28c7-49f7-9880-46c82f64a7ee None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:05.578216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e08fc9-28c7-49f7-9880-46c82f64a7ee None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:05.578487 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e08fc9-28c7-49f7-9880-46c82f64a7ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:05.579606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e08fc9-28c7-49f7-9880-46c82f64a7ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:05.581077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e08fc9-28c7-49f7-9880-46c82f64a7ee None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:05.582722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 28/56] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:05.588349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c211a6-f8c2-4e95-be38-b367b4eea216 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:05.689153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c211a6-f8c2-4e95-be38-b367b4eea216 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:05.703779 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:05.723366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04773a9d-5a97-48d5-9aee-e8b5ac536a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:05.723978 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04773a9d-5a97-48d5-9aee-e8b5ac536a76 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:05.724259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04773a9d-5a97-48d5-9aee-e8b5ac536a76 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:05.724534 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04773a9d-5a97-48d5-9aee-e8b5ac536a76 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:05.725168 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04773a9d-5a97-48d5-9aee-e8b5ac536a76 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:05.726633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04773a9d-5a97-48d5-9aee-e8b5ac536a76 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:05.815891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 29/57] 10.222.0.42 () {60 vars in 1140 bytes} [Wed Sep 14 19:33:05 2022] POST /identity/v3/projects => generated 298 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:33:07.939045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ad4c3-230f-4f90-bbb6-2b5d360032fc None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:07.939498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ad4c3-230f-4f90-bbb6-2b5d360032fc None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:07.939611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ad4c3-230f-4f90-bbb6-2b5d360032fc None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:07.940954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 29/58] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:07.946400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e617d-0e38-4c7e-be94-12885aa3bc50 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:07.946806 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e617d-0e38-4c7e-be94-12885aa3bc50 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:07.946947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54e617d-0e38-4c7e-be94-12885aa3bc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:07.964975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f54e617d-0e38-4c7e-be94-12885aa3bc50 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:07.966182 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f54e617d-0e38-4c7e-be94-12885aa3bc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:08.029354 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 30/59] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:08.293064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ad4c3-230f-4f90-bbb6-2b5d360032fc None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:08.391468 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ad4c3-230f-4f90-bbb6-2b5d360032fc None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:08.423964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b97174-11e0-4b3d-80fa-f26e78b905e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:08.424649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b97174-11e0-4b3d-80fa-f26e78b905e4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:08.424940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b97174-11e0-4b3d-80fa-f26e78b905e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:08.425219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b97174-11e0-4b3d-80fa-f26e78b905e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:08.426350 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b97174-11e0-4b3d-80fa-f26e78b905e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:08.427779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b97174-11e0-4b3d-80fa-f26e78b905e4 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:08.429315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 30/60] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:08.434715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e617d-0e38-4c7e-be94-12885aa3bc50 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:08.523090 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:08.528404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54e617d-0e38-4c7e-be94-12885aa3bc50 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:08.560455 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7cad-17f4-41b2-935f-ce52b54f33c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:08.560996 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7cad-17f4-41b2-935f-ce52b54f33c1 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:08.561273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7cad-17f4-41b2-935f-ce52b54f33c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:08.561578 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7cad-17f4-41b2-935f-ce52b54f33c1 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:08.562246 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e7cad-17f4-41b2-935f-ce52b54f33c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:08.563669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e7cad-17f4-41b2-935f-ce52b54f33c1 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:08.675021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 31/61] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:33:08 2022] POST /identity/v3/users => generated 275 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:33:11.954395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49437b3-bdcb-4c74-a61c-55ecedd71a82 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:11.954395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49437b3-bdcb-4c74-a61c-55ecedd71a82 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:11.954395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49437b3-bdcb-4c74-a61c-55ecedd71a82 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:11.954395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 31/62] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:11.960543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812fd3a-c065-4727-a62b-533d60843734 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:11.961349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812fd3a-c065-4727-a62b-533d60843734 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:11.961842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812fd3a-c065-4727-a62b-533d60843734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:11.990813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c812fd3a-c065-4727-a62b-533d60843734 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:11.992380 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c812fd3a-c065-4727-a62b-533d60843734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:12.008007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2cdba-6582-42b6-b862-c77c25de4d00 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:12.008594 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2cdba-6582-42b6-b862-c77c25de4d00 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:12.009066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2cdba-6582-42b6-b862-c77c25de4d00 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:12.010651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 32/63] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:33:12.019788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70900a5e-cb81-48a9-8d4e-17b6ff569b7b None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:12.020321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70900a5e-cb81-48a9-8d4e-17b6ff569b7b None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:12.020810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70900a5e-cb81-48a9-8d4e-17b6ff569b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:12.045738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-70900a5e-cb81-48a9-8d4e-17b6ff569b7b None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:12.049220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-70900a5e-cb81-48a9-8d4e-17b6ff569b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:12.089217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:12.150188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 33/64] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:12.172601 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 32/65] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:12.330223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa896e-ef20-4d21-ab08-72681ae9a343 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:12.330852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa896e-ef20-4d21-ab08-72681ae9a343 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:12.331332 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0aa896e-ef20-4d21-ab08-72681ae9a343 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:12.333163 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 34/66] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:12.338188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f04cae-ce21-48e4-8fa6-cf97df7f9ea6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:12.338474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f04cae-ce21-48e4-8fa6-cf97df7f9ea6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:12.338752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f04cae-ce21-48e4-8fa6-cf97df7f9ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:12.366784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e2f04cae-ce21-48e4-8fa6-cf97df7f9ea6 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:12.367965 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e2f04cae-ce21-48e4-8fa6-cf97df7f9ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:12.468582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 33/67] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:12.495075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa896e-ef20-4d21-ab08-72681ae9a343 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:12.510840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f04cae-ce21-48e4-8fa6-cf97df7f9ea6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:12.533745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:12.649305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f04cae-ce21-48e4-8fa6-cf97df7f9ea6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:12.687343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df1e054-a593-4b2d-8c25-5ef0c1abb218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:12.688400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1e054-a593-4b2d-8c25-5ef0c1abb218 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:12.688948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1e054-a593-4b2d-8c25-5ef0c1abb218 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:12.689432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df1e054-a593-4b2d-8c25-5ef0c1abb218 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:12.694508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0aa896e-ef20-4d21-ab08-72681ae9a343 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:12.709679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df1e054-a593-4b2d-8c25-5ef0c1abb218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-4df1e054-a593-4b2d-8c25-5ef0c1abb218 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:12.720761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:12.721609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:12.722449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:12.722449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:12.722449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:12.722449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:12.722449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:12.722449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:12.722449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:12.722449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:12.722449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:12.722449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:12.722449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:12.722449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:12.726878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 34/68] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:33:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:33:12.728030 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ac027-7b5d-4dda-890d-fe49254bbb64 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:12.728314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ac027-7b5d-4dda-890d-fe49254bbb64 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:12.728583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ac027-7b5d-4dda-890d-fe49254bbb64 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:12.729877 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 35/69] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:12.735330 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ac027-7b5d-4dda-890d-fe49254bbb64 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:12.759386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751781f-0a83-450e-a310-8f4837c21d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:12.761069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751781f-0a83-450e-a310-8f4837c21d14 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:12.761938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751781f-0a83-450e-a310-8f4837c21d14 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:12.762423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751781f-0a83-450e-a310-8f4837c21d14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:12.779081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751781f-0a83-450e-a310-8f4837c21d14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-3751781f-0a83-450e-a310-8f4837c21d14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:12.784412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:12.785342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:12.786150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:12.786150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:12.786150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:12.786150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:12.786150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:12.786150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:12.786150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:12.786150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:12.786150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:12.786150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:12.786150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:12.786150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:12.786150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 35/70] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:12.786952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0eaea-bcba-400a-8504-7f6c5a51127e None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:12.787109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0eaea-bcba-400a-8504-7f6c5a51127e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:12.792699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c0eaea-bcba-400a-8504-7f6c5a51127e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:12.809210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-30c0eaea-bcba-400a-8504-7f6c5a51127e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:12.810421 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-30c0eaea-bcba-400a-8504-7f6c5a51127e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:12.824190 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:12.839663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:12.880532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ac027-7b5d-4dda-890d-fe49254bbb64 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:12.887124 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 36/71] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:12.887964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0eaea-bcba-400a-8504-7f6c5a51127e None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:12.915066 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d1b1e-a235-440d-b2d0-8af214226069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:12.916112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d1b1e-a235-440d-b2d0-8af214226069 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:12.916530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d1b1e-a235-440d-b2d0-8af214226069 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:12.916805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d1b1e-a235-440d-b2d0-8af214226069 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:12.917462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d1b1e-a235-440d-b2d0-8af214226069 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:12.917760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d1b1e-a235-440d-b2d0-8af214226069 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:12.919266 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d1b1e-a235-440d-b2d0-8af214226069 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:12.934172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 36/72] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:12.934172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d1b1e-a235-440d-b2d0-8af214226069 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:12.987617 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c0eaea-bcba-400a-8504-7f6c5a51127e None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:13.021665 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8791592-5d35-4414-ab41-7a64e7fe87fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:13.022306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791592-5d35-4414-ab41-7a64e7fe87fe None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:13.022583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791592-5d35-4414-ab41-7a64e7fe87fe None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:13.022852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8791592-5d35-4414-ab41-7a64e7fe87fe None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:13.023504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8791592-5d35-4414-ab41-7a64e7fe87fe None admin] RBAC: Adding query filter params (name=member) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:13.023805 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8791592-5d35-4414-ab41-7a64e7fe87fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:13.025180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8791592-5d35-4414-ab41-7a64e7fe87fe None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:13.034291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 37/73] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:13.039996 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578cd09-37ac-4a98-aed4-77d1afdc66bb None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:13.040285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578cd09-37ac-4a98-aed4-77d1afdc66bb None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:13.040564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d578cd09-37ac-4a98-aed4-77d1afdc66bb None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:13.041954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 38/74] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:13.042483 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578cd09-37ac-4a98-aed4-77d1afdc66bb None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:13.048503 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d1b1e-a235-440d-b2d0-8af214226069 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:13.055957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:13.086111 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dad8e6-349b-4d7a-bb17-cac7386c9440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:13.086803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dad8e6-349b-4d7a-bb17-cac7386c9440 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:13.087084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dad8e6-349b-4d7a-bb17-cac7386c9440 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:13.087359 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dad8e6-349b-4d7a-bb17-cac7386c9440 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:13.089287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:13.100853 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dad8e6-349b-4d7a-bb17-cac7386c9440 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-51dad8e6-349b-4d7a-bb17-cac7386c9440 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:13.105784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:13.106665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:13.110700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:13.110700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:13.110700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:13.110700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:13.110700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:13.110700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:13.110700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:13.110700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:13.110700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:13.110700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:13.110700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:13.110700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:13.110700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 37/75] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:33:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:13.110700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dad8e6-349b-4d7a-bb17-cac7386c9440 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:13.158398 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d578cd09-37ac-4a98-aed4-77d1afdc66bb None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:13.193173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b83dff-e0ff-460e-89cd-0eb29aeda649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:13.193811 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83dff-e0ff-460e-89cd-0eb29aeda649 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:13.194123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83dff-e0ff-460e-89cd-0eb29aeda649 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:13.194404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b83dff-e0ff-460e-89cd-0eb29aeda649 None admin] PATH_INFO: `/v3/users/e8f45104d2d540658c4bbc32613a9317` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:13.195535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b83dff-e0ff-460e-89cd-0eb29aeda649 None admin] RBAC: Authorizing `identity:get_user(user_id=e8f45104d2d540658c4bbc32613a9317)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:13.196982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b83dff-e0ff-460e-89cd-0eb29aeda649 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:13.198597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 39/76] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:13 2022] GET /identity/v3/users/e8f45104d2d540658c4bbc32613a9317 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:13.203916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37925b51-3a98-4544-b10a-4ef115e25393 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:13.203916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37925b51-3a98-4544-b10a-4ef115e25393 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:13.203916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37925b51-3a98-4544-b10a-4ef115e25393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:13.216418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dad8e6-349b-4d7a-bb17-cac7386c9440 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:13.218302 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-37925b51-3a98-4544-b10a-4ef115e25393 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:13.219543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-37925b51-3a98-4544-b10a-4ef115e25393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:13.249837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891e7cf-fab7-4fea-8c5b-6a954607b113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:13.250409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891e7cf-fab7-4fea-8c5b-6a954607b113 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:13.250693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891e7cf-fab7-4fea-8c5b-6a954607b113 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:13.250967 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1891e7cf-fab7-4fea-8c5b-6a954607b113 None admin] PATH_INFO: `/v3/users/34554fb083404728bd4ff4fa2b4a8149` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:13.266278 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891e7cf-fab7-4fea-8c5b-6a954607b113 None admin] RBAC: Authorizing `identity:get_user(user_id=34554fb083404728bd4ff4fa2b4a8149)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:13.267735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1891e7cf-fab7-4fea-8c5b-6a954607b113 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:13.269351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 38/77] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:13 2022] GET /identity/v3/users/34554fb083404728bd4ff4fa2b4a8149 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:13.270143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891e7cf-fab7-4fea-8c5b-6a954607b113 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:13.287345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 40/78] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:13.288185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37925b51-3a98-4544-b10a-4ef115e25393 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:13.296068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:13.328162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:13.381858 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1891e7cf-fab7-4fea-8c5b-6a954607b113 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:13.393821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37925b51-3a98-4544-b10a-4ef115e25393 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:13.415836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293ea34-83f7-4223-88bf-7c01d41b12f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:13.416449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293ea34-83f7-4223-88bf-7c01d41b12f9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:13.416735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293ea34-83f7-4223-88bf-7c01d41b12f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:13.417013 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293ea34-83f7-4223-88bf-7c01d41b12f9 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:13.417667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293ea34-83f7-4223-88bf-7c01d41b12f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:13.418003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293ea34-83f7-4223-88bf-7c01d41b12f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:13.419379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4293ea34-83f7-4223-88bf-7c01d41b12f9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:13.427895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 39/79] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:33:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:13.428684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293ea34-83f7-4223-88bf-7c01d41b12f9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:13.429690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394c398-95cc-4311-8476-171a1f0067a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:13.430353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394c398-95cc-4311-8476-171a1f0067a2 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:13.430634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394c398-95cc-4311-8476-171a1f0067a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:13.430908 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394c398-95cc-4311-8476-171a1f0067a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:13.441343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a394c398-95cc-4311-8476-171a1f0067a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-a394c398-95cc-4311-8476-171a1f0067a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:13.446517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:13.447560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:13.448564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:13.448564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:13.448564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:13.448564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:13.448564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:13.448564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:13.448564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:13.448564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:13.448564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:13.448564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:13.448564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:13.448564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:13.448564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 41/80] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:13.453709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394c398-95cc-4311-8476-171a1f0067a2 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:13.520311 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:13.541106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293ea34-83f7-4223-88bf-7c01d41b12f9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:13.556324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:13.556820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394c398-95cc-4311-8476-171a1f0067a2 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:13.577492 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bed00d-c210-440a-a9e7-19f0012691c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:13.578140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bed00d-c210-440a-a9e7-19f0012691c3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:13.578423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bed00d-c210-440a-a9e7-19f0012691c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:13.578702 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bed00d-c210-440a-a9e7-19f0012691c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:13.592245 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0c23c-da22-483b-9f17-93bfc1290934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:13.592883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0c23c-da22-483b-9f17-93bfc1290934 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:13.593164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0c23c-da22-483b-9f17-93bfc1290934 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:13.593441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d0c23c-da22-483b-9f17-93bfc1290934 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:13.643409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bed00d-c210-440a-a9e7-19f0012691c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:13.644871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bed00d-c210-440a-a9e7-19f0012691c3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:13.659266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0c23c-da22-483b-9f17-93bfc1290934 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:13.660716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d0c23c-da22-483b-9f17-93bfc1290934 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:13.723013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:13.741312 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 40/81] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:13.742338 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bed00d-c210-440a-a9e7-19f0012691c3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:13.758764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:13.762502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 42/82] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:13.763336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0c23c-da22-483b-9f17-93bfc1290934 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:13.847149 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bed00d-c210-440a-a9e7-19f0012691c3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:13.862607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d0c23c-da22-483b-9f17-93bfc1290934 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:13.882488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae2547-8925-4bd1-91a2-7fb1eaa28f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:13.883126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae2547-8925-4bd1-91a2-7fb1eaa28f49 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:13.883410 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae2547-8925-4bd1-91a2-7fb1eaa28f49 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:13.883684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae2547-8925-4bd1-91a2-7fb1eaa28f49 None admin] PATH_INFO: `/v3/users/34554fb083404728bd4ff4fa2b4a8149` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:13.884815 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae2547-8925-4bd1-91a2-7fb1eaa28f49 None admin] RBAC: Authorizing `identity:get_user(user_id=34554fb083404728bd4ff4fa2b4a8149)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:13.886304 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ae2547-8925-4bd1-91a2-7fb1eaa28f49 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:13.887884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 41/83] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:13 2022] GET /identity/v3/users/34554fb083404728bd4ff4fa2b4a8149 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:13.893082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae2547-8925-4bd1-91a2-7fb1eaa28f49 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:13.896264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac2139-f0ff-4a08-ad5b-35ed4da838d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:13.896872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac2139-f0ff-4a08-ad5b-35ed4da838d3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:13.897148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac2139-f0ff-4a08-ad5b-35ed4da838d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:13.897416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac2139-f0ff-4a08-ad5b-35ed4da838d3 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:13.898098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac2139-f0ff-4a08-ad5b-35ed4da838d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:13.898397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac2139-f0ff-4a08-ad5b-35ed4da838d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:13.899759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac2139-f0ff-4a08-ad5b-35ed4da838d3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:13.908198 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 43/84] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:13.912813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac2139-f0ff-4a08-ad5b-35ed4da838d3 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:13.936554 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:13.974439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:13.992009 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ae2547-8925-4bd1-91a2-7fb1eaa28f49 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:14.007202 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac2139-f0ff-4a08-ad5b-35ed4da838d3 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:14.025225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc417b-c7d4-4fd0-b0bd-f1c6ba84a7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:14.025878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc417b-c7d4-4fd0-b0bd-f1c6ba84a7a3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:14.026158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc417b-c7d4-4fd0-b0bd-f1c6ba84a7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:14.026427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc417b-c7d4-4fd0-b0bd-f1c6ba84a7a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:14.027507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc417b-c7d4-4fd0-b0bd-f1c6ba84a7a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:14.028941 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc417b-c7d4-4fd0-b0bd-f1c6ba84a7a3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:14.031275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 42/85] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:14.031818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc417b-c7d4-4fd0-b0bd-f1c6ba84a7a3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:14.042187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27723d18-f0b8-4e1f-9d7b-471f23daae25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:14.042821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27723d18-f0b8-4e1f-9d7b-471f23daae25 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:14.043103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27723d18-f0b8-4e1f-9d7b-471f23daae25 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:14.043379 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27723d18-f0b8-4e1f-9d7b-471f23daae25 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:14.044493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27723d18-f0b8-4e1f-9d7b-471f23daae25 None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:14.046580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27723d18-f0b8-4e1f-9d7b-471f23daae25 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:14.053041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 44/86] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:13 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:14.053041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27723d18-f0b8-4e1f-9d7b-471f23daae25 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:14.129923 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc417b-c7d4-4fd0-b0bd-f1c6ba84a7a3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:14.140941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:14.148782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27723d18-f0b8-4e1f-9d7b-471f23daae25 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:14.167188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804162fe-7d85-4bec-b05a-1ec265fbd7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:14.167834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804162fe-7d85-4bec-b05a-1ec265fbd7c8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:14.168117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804162fe-7d85-4bec-b05a-1ec265fbd7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:14.168389 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804162fe-7d85-4bec-b05a-1ec265fbd7c8 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:14.169549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804162fe-7d85-4bec-b05a-1ec265fbd7c8 None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:14.171108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804162fe-7d85-4bec-b05a-1ec265fbd7c8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:14.172744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 43/87] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:14 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:14.178702 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804162fe-7d85-4bec-b05a-1ec265fbd7c8 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:14.182196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d6e13-2634-4983-9c59-2d4f0b174e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:14.182801 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d6e13-2634-4983-9c59-2d4f0b174e0e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:14.183080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d6e13-2634-4983-9c59-2d4f0b174e0e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:14.183350 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d6e13-2634-4983-9c59-2d4f0b174e0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:14.196846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:14.248330 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d6e13-2634-4983-9c59-2d4f0b174e0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:14.250408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d6e13-2634-4983-9c59-2d4f0b174e0e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:14.275702 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804162fe-7d85-4bec-b05a-1ec265fbd7c8 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:14.308826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5448e3-10d9-4f14-bde0-1b20afeb1c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:14.309447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5448e3-10d9-4f14-bde0-1b20afeb1c87 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:14.309725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5448e3-10d9-4f14-bde0-1b20afeb1c87 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:14.310063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5448e3-10d9-4f14-bde0-1b20afeb1c87 None admin] PATH_INFO: `/v3/users/34554fb083404728bd4ff4fa2b4a8149` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:14.311187 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5448e3-10d9-4f14-bde0-1b20afeb1c87 None admin] RBAC: Authorizing `identity:get_user(user_id=34554fb083404728bd4ff4fa2b4a8149)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:14.313168 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5448e3-10d9-4f14-bde0-1b20afeb1c87 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:14.314742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 44/88] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:14 2022] GET /identity/v3/users/34554fb083404728bd4ff4fa2b4a8149 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:14.318647 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5448e3-10d9-4f14-bde0-1b20afeb1c87 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:14.339350 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:14.345592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 45/89] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:14.346429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d6e13-2634-4983-9c59-2d4f0b174e0e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:14.401355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:14.415868 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5448e3-10d9-4f14-bde0-1b20afeb1c87 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:14.441847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d6e13-2634-4983-9c59-2d4f0b174e0e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:14.449629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955377d-6e12-4f26-902f-aeb509602447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:14.450266 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955377d-6e12-4f26-902f-aeb509602447 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:14.450553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955377d-6e12-4f26-902f-aeb509602447 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:14.450831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955377d-6e12-4f26-902f-aeb509602447 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:14.451473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955377d-6e12-4f26-902f-aeb509602447 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:14.452947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955377d-6e12-4f26-902f-aeb509602447 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:14.475234 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5df0ea-ff14-45ef-811c-3c97be22f694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:14.475837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5df0ea-ff14-45ef-811c-3c97be22f694 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:14.476121 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5df0ea-ff14-45ef-811c-3c97be22f694 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:14.476400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5df0ea-ff14-45ef-811c-3c97be22f694 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:14.477121 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5df0ea-ff14-45ef-811c-3c97be22f694 None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=07768f903a224486a253fc7acc9c3360, user.id=e8f45104d2d540658c4bbc32613a9317) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:14.477435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5df0ea-ff14-45ef-811c-3c97be22f694 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:14.478907 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5df0ea-ff14-45ef-811c-3c97be22f694 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:14.484827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 46/90] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:14 2022] GET /identity/v3/role_assignments?user.id=e8f45104d2d540658c4bbc32613a9317&scope.project.id=07768f903a224486a253fc7acc9c3360&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:14.485599 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5df0ea-ff14-45ef-811c-3c97be22f694 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:14.548714 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:14.590933 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5df0ea-ff14-45ef-811c-3c97be22f694 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:14.604208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 45/91] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:33:14 2022] POST /identity/v3/users => generated 289 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:33:14.604870 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955377d-6e12-4f26-902f-aeb509602447 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:14.642214 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecbe74-9153-49d2-8440-514b6a4afcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:14.643210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecbe74-9153-49d2-8440-514b6a4afcfb None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:14.643697 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecbe74-9153-49d2-8440-514b6a4afcfb None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:14.643976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fecbe74-9153-49d2-8440-514b6a4afcfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:14.644735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecbe74-9153-49d2-8440-514b6a4afcfb None admin] RBAC: Adding query filter params (role.id=efa4ca02b2ff4ad6b14d501d4d79bbea, scope.project.id=07768f903a224486a253fc7acc9c3360, user.id=34554fb083404728bd4ff4fa2b4a8149) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:14.645049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecbe74-9153-49d2-8440-514b6a4afcfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:14.646931 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fecbe74-9153-49d2-8440-514b6a4afcfb None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:14.655799 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 47/92] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:14 2022] GET /identity/v3/role_assignments?user.id=34554fb083404728bd4ff4fa2b4a8149&scope.project.id=07768f903a224486a253fc7acc9c3360&role.id=efa4ca02b2ff4ad6b14d501d4d79bbea => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:14.659419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecbe74-9153-49d2-8440-514b6a4afcfb None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:14.805434 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fecbe74-9153-49d2-8440-514b6a4afcfb None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:14.816043 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955377d-6e12-4f26-902f-aeb509602447 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:14.835066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:14.851977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f850ef-faef-4f2e-b6d2-3757ed0d8d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:14.852860 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f850ef-faef-4f2e-b6d2-3757ed0d8d6a None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:14.853580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f850ef-faef-4f2e-b6d2-3757ed0d8d6a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:14.853909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f850ef-faef-4f2e-b6d2-3757ed0d8d6a None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:14.855497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f850ef-faef-4f2e-b6d2-3757ed0d8d6a None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:14.856993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f850ef-faef-4f2e-b6d2-3757ed0d8d6a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:14.859187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 48/93] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:14 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:14.865220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f850ef-faef-4f2e-b6d2-3757ed0d8d6a None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:14.871954 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b77f7d-df6a-48d4-8f0d-1b29b6b80ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:14.872600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b77f7d-df6a-48d4-8f0d-1b29b6b80ad9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:14.872883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b77f7d-df6a-48d4-8f0d-1b29b6b80ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:14.873159 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b77f7d-df6a-48d4-8f0d-1b29b6b80ad9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:14.888050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:14.959113 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b77f7d-df6a-48d4-8f0d-1b29b6b80ad9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:14.960871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b77f7d-df6a-48d4-8f0d-1b29b6b80ad9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:15.004564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f850ef-faef-4f2e-b6d2-3757ed0d8d6a None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:15.054604 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e93578-95a2-4314-b676-852a7e25301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:15.056419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e93578-95a2-4314-b676-852a7e25301a None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:15.056846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e93578-95a2-4314-b676-852a7e25301a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:15.057662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e93578-95a2-4314-b676-852a7e25301a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:15.058845 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e93578-95a2-4314-b676-852a7e25301a None admin] RBAC: Adding query filter params (role.id=36b107b4892f47f8af6b65e1d1d94bb0, scope.project.id=07768f903a224486a253fc7acc9c3360, user.id=34554fb083404728bd4ff4fa2b4a8149) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:15.059314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e93578-95a2-4314-b676-852a7e25301a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:15.061271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e93578-95a2-4314-b676-852a7e25301a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:15.072249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 49/94] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:14 2022] GET /identity/v3/role_assignments?user.id=34554fb083404728bd4ff4fa2b4a8149&scope.project.id=07768f903a224486a253fc7acc9c3360&role.id=36b107b4892f47f8af6b65e1d1d94bb0 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:15.086595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 46/95] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:15.092804 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e93578-95a2-4314-b676-852a7e25301a None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:15.159166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:15.209102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e93578-95a2-4314-b676-852a7e25301a None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:15.253081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f360f57-0cda-4e2a-9b42-9bc1243f06b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:15.253650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f360f57-0cda-4e2a-9b42-9bc1243f06b6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:15.254003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f360f57-0cda-4e2a-9b42-9bc1243f06b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:15.254280 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f360f57-0cda-4e2a-9b42-9bc1243f06b6 None admin] PATH_INFO: `/v3/projects/eff97a5d26c54873ae5acd2f1d454e39` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:15.255451 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f360f57-0cda-4e2a-9b42-9bc1243f06b6 None admin] RBAC: Authorizing `identity:get_project(project_id=eff97a5d26c54873ae5acd2f1d454e39)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:15.256940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f360f57-0cda-4e2a-9b42-9bc1243f06b6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:15.259074 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 50/96] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:15 2022] GET /identity/v3/projects/eff97a5d26c54873ae5acd2f1d454e39 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:15.267008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b77f7d-df6a-48d4-8f0d-1b29b6b80ad9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:15.314123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:15.383357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b77f7d-df6a-48d4-8f0d-1b29b6b80ad9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:15.419300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276de939-0f60-4887-b742-26322c2f8ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:15.419914 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276de939-0f60-4887-b742-26322c2f8ce5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:15.420193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276de939-0f60-4887-b742-26322c2f8ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:15.420469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276de939-0f60-4887-b742-26322c2f8ce5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:15.421335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276de939-0f60-4887-b742-26322c2f8ce5 None admin] RBAC: Adding query filter params (role.id=efa4ca02b2ff4ad6b14d501d4d79bbea, scope.project.id=eff97a5d26c54873ae5acd2f1d454e39, user.id=34554fb083404728bd4ff4fa2b4a8149) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:15.421640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276de939-0f60-4887-b742-26322c2f8ce5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:15.423109 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276de939-0f60-4887-b742-26322c2f8ce5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:15.432574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 47/97] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:15 2022] GET /identity/v3/role_assignments?user.id=34554fb083404728bd4ff4fa2b4a8149&scope.project.id=eff97a5d26c54873ae5acd2f1d454e39&role.id=efa4ca02b2ff4ad6b14d501d4d79bbea => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:17.119776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d5a52-676e-48cd-8280-b8c0634a5f66 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:17.120800 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d5a52-676e-48cd-8280-b8c0634a5f66 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:17.121282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555d5a52-676e-48cd-8280-b8c0634a5f66 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:17.123211 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 51/98] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:17.128165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bd029-4933-4f6d-b500-80d45b0942db None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:17.128977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bd029-4933-4f6d-b500-80d45b0942db None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:17.129452 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263bd029-4933-4f6d-b500-80d45b0942db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:17.153997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-263bd029-4933-4f6d-b500-80d45b0942db None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:17.155542 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-263bd029-4933-4f6d-b500-80d45b0942db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:17.236763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 48/99] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:17.418469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58016058-79db-4331-ac80-6da0bde9ce64 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:17.419091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58016058-79db-4331-ac80-6da0bde9ce64 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:17.419565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58016058-79db-4331-ac80-6da0bde9ce64 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:17.422000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 52/100] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:17.427494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8837682-619e-4858-bbfa-6034ddd17900 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:17.428037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8837682-619e-4858-bbfa-6034ddd17900 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:17.428510 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8837682-619e-4858-bbfa-6034ddd17900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:17.454293 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c8837682-619e-4858-bbfa-6034ddd17900 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:17.455801 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c8837682-619e-4858-bbfa-6034ddd17900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:17.491053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:17.544099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 49/101] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:17.672813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58016058-79db-4331-ac80-6da0bde9ce64 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:17.800660 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58016058-79db-4331-ac80-6da0bde9ce64 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:17.824975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:17.836296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e5e59-7038-43c0-98e7-104f784e3b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:17.836934 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e5e59-7038-43c0-98e7-104f784e3b78 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:17.866638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e5e59-7038-43c0-98e7-104f784e3b78 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:17.866638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e5e59-7038-43c0-98e7-104f784e3b78 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:17.866638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e5e59-7038-43c0-98e7-104f784e3b78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:17.866638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-9d4e5e59-7038-43c0-98e7-104f784e3b78 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:17.866638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:17.866638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:17.866638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:17.866638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:17.866638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:17.866638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:17.867215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:17.868096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:17.868884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:17.868884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:17.868884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:17.868884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 53/102] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:33:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:17.869096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8837682-619e-4858-bbfa-6034ddd17900 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:17.984983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8837682-619e-4858-bbfa-6034ddd17900 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:18.001846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e5e59-7038-43c0-98e7-104f784e3b78 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:18.030250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9d63a-4a16-4015-b7f3-4877eeb695fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:18.030878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9d63a-4a16-4015-b7f3-4877eeb695fa None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:18.031155 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9d63a-4a16-4015-b7f3-4877eeb695fa None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:18.031431 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9d63a-4a16-4015-b7f3-4877eeb695fa None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:18.032087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9d63a-4a16-4015-b7f3-4877eeb695fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:18.032385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9d63a-4a16-4015-b7f3-4877eeb695fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:18.033754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f9d63a-4a16-4015-b7f3-4877eeb695fa None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:18.053983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 50/103] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:18.063792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9d63a-4a16-4015-b7f3-4877eeb695fa None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:18.108193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:18.189036 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9d63a-4a16-4015-b7f3-4877eeb695fa None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:18.229881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b912ef-cdac-4494-976e-bc5f14121c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:18.230517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b912ef-cdac-4494-976e-bc5f14121c18 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:18.230796 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b912ef-cdac-4494-976e-bc5f14121c18 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:18.231070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b912ef-cdac-4494-976e-bc5f14121c18 None admin] PATH_INFO: `/v3/users/e8f45104d2d540658c4bbc32613a9317` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:18.232721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b912ef-cdac-4494-976e-bc5f14121c18 None admin] RBAC: Authorizing `identity:get_user(user_id=e8f45104d2d540658c4bbc32613a9317)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:18.234267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b912ef-cdac-4494-976e-bc5f14121c18 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:18.236075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 51/104] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:18 2022] GET /identity/v3/users/e8f45104d2d540658c4bbc32613a9317 => generated 247 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:18.242373 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e5e59-7038-43c0-98e7-104f784e3b78 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:18.250839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b912ef-cdac-4494-976e-bc5f14121c18 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:18.335003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76a552-8274-4ca1-a359-960bc97ebc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:18.336114 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76a552-8274-4ca1-a359-960bc97ebc2e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:18.336590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76a552-8274-4ca1-a359-960bc97ebc2e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:18.337071 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76a552-8274-4ca1-a359-960bc97ebc2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:18.362972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76a552-8274-4ca1-a359-960bc97ebc2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-6b76a552-8274-4ca1-a359-960bc97ebc2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:18.374469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:18.375360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:18.376132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:18.376132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:18.376132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:18.376132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:18.376132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:18.376132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:18.376132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:18.376132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:18.376132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:18.376132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:18.376132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:18.376132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:18.378774 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 54/105] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:18.384255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76a552-8274-4ca1-a359-960bc97ebc2e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:18.413276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:18.426145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b912ef-cdac-4494-976e-bc5f14121c18 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:18.466482 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85b9a84-e155-4433-a2ef-dca49c617b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:18.469821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b9a84-e155-4433-a2ef-dca49c617b13 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:18.469821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b9a84-e155-4433-a2ef-dca49c617b13 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:18.469821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85b9a84-e155-4433-a2ef-dca49c617b13 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:18.469821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b9a84-e155-4433-a2ef-dca49c617b13 None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:18.470700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85b9a84-e155-4433-a2ef-dca49c617b13 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:18.475889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 52/106] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:18 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:18.475889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c411889-d57a-49df-9bce-fb9b6e98c742 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:18.475889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c411889-d57a-49df-9bce-fb9b6e98c742 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:18.475889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c411889-d57a-49df-9bce-fb9b6e98c742 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:18.475889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 53/107] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:18.484260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:18.485784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c560a5-b154-444e-bef2-39a65fad6ca5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:18.486369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c560a5-b154-444e-bef2-39a65fad6ca5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:18.486588 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c560a5-b154-444e-bef2-39a65fad6ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:18.527506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e8c560a5-b154-444e-bef2-39a65fad6ca5 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:18.528880 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e8c560a5-b154-444e-bef2-39a65fad6ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:18.588907 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76a552-8274-4ca1-a359-960bc97ebc2e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:18.628392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 54/108] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:18.628392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c560a5-b154-444e-bef2-39a65fad6ca5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:18.636661 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa667e2-a134-4904-bdc1-eb21d39fe096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:18.637276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa667e2-a134-4904-bdc1-eb21d39fe096 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:18.637561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa667e2-a134-4904-bdc1-eb21d39fe096 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:18.637865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa667e2-a134-4904-bdc1-eb21d39fe096 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:18.638547 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa667e2-a134-4904-bdc1-eb21d39fe096 None admin] RBAC: Adding query filter params (name=member) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:18.638852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa667e2-a134-4904-bdc1-eb21d39fe096 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:18.640781 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa667e2-a134-4904-bdc1-eb21d39fe096 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:18.650785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 55/109] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:18.652111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f1680-03c1-4341-9007-74c67713eeb1 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:18.652405 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f1680-03c1-4341-9007-74c67713eeb1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:18.652682 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f1680-03c1-4341-9007-74c67713eeb1 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:18.654341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 56/110] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:18.655555 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcce21a-6023-413b-b5b1-3ca78c015a02 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:18.655871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcce21a-6023-413b-b5b1-3ca78c015a02 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:18.656225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcce21a-6023-413b-b5b1-3ca78c015a02 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:18.661513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 57/111] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:18.662319 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcce21a-6023-413b-b5b1-3ca78c015a02 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:18.752344 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:18.755894 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c560a5-b154-444e-bef2-39a65fad6ca5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:18.774140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcce21a-6023-413b-b5b1-3ca78c015a02 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:18.783485 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:18.797126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a6996-2cd4-404e-9561-e1f72c1a8be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:18.797819 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a6996-2cd4-404e-9561-e1f72c1a8be9 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:18.798096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a6996-2cd4-404e-9561-e1f72c1a8be9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:18.798370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a6996-2cd4-404e-9561-e1f72c1a8be9 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360/users/e8f45104d2d540658c4bbc32613a9317/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:18.801047 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a6996-2cd4-404e-9561-e1f72c1a8be9 None admin] RBAC: Authorizing `identity:create_grant(project_id=07768f903a224486a253fc7acc9c3360, user_id=e8f45104d2d540658c4bbc32613a9317, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:18.802531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a6996-2cd4-404e-9561-e1f72c1a8be9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:18.813674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a3cf6b-f8aa-4993-b905-e343707fa7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:18.819365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3cf6b-f8aa-4993-b905-e343707fa7bd None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:18.819365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3cf6b-f8aa-4993-b905-e343707fa7bd None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:18.819365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a3cf6b-f8aa-4993-b905-e343707fa7bd None admin] PATH_INFO: `/v3/users/34554fb083404728bd4ff4fa2b4a8149` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:18.819365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3cf6b-f8aa-4993-b905-e343707fa7bd None admin] RBAC: Authorizing `identity:get_user(user_id=34554fb083404728bd4ff4fa2b4a8149)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:18.819365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a3cf6b-f8aa-4993-b905-e343707fa7bd None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:18.819919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 58/112] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:18 2022] GET /identity/v3/users/34554fb083404728bd4ff4fa2b4a8149 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:18.821109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab21ff-1f22-4f3b-b1b3-f9e28eaab972 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:18.821442 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab21ff-1f22-4f3b-b1b3-f9e28eaab972 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:18.821718 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab21ff-1f22-4f3b-b1b3-f9e28eaab972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:18.843648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e2ab21ff-1f22-4f3b-b1b3-f9e28eaab972 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:18.848375 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e2ab21ff-1f22-4f3b-b1b3-f9e28eaab972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:18.881098 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 55/113] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:33:18 2022] PUT /identity/v3/projects/07768f903a224486a253fc7acc9c3360/users/e8f45104d2d540658c4bbc32613a9317/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:18.882438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7328175-7d90-4ad5-bc20-47f6ffc644cf None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:18.883071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7328175-7d90-4ad5-bc20-47f6ffc644cf None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:18.883604 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7328175-7d90-4ad5-bc20-47f6ffc644cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:18.906469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d7328175-7d90-4ad5-bc20-47f6ffc644cf None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:18.908123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d7328175-7d90-4ad5-bc20-47f6ffc644cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:18.938569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 59/114] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:18.939507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab21ff-1f22-4f3b-b1b3-f9e28eaab972 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:19.002107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 56/115] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:19.004139 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36626f32-d99a-4778-9df5-e6ad7dbf7755 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:19.004369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36626f32-d99a-4778-9df5-e6ad7dbf7755 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:19.004647 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36626f32-d99a-4778-9df5-e6ad7dbf7755 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:19.006459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 57/116] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:19.007225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36626f32-d99a-4778-9df5-e6ad7dbf7755 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:19.080133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab21ff-1f22-4f3b-b1b3-f9e28eaab972 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:19.083975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:19.112881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:19.129508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f676f5a1-ab43-433d-9f8d-429a19d382a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:19.130204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676f5a1-ab43-433d-9f8d-429a19d382a9 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:19.130504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676f5a1-ab43-433d-9f8d-429a19d382a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:19.131082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f676f5a1-ab43-433d-9f8d-429a19d382a9 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:19.132491 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676f5a1-ab43-433d-9f8d-429a19d382a9 None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:19.133982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f676f5a1-ab43-433d-9f8d-429a19d382a9 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:19.134764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36626f32-d99a-4778-9df5-e6ad7dbf7755 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:19.135674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 60/117] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:18 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:19.137068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fee82e-f507-4b05-bf66-d9e402d1f79e None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:19.137355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fee82e-f507-4b05-bf66-d9e402d1f79e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:19.137633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fee82e-f507-4b05-bf66-d9e402d1f79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:19.166357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-63fee82e-f507-4b05-bf66-d9e402d1f79e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:19.167759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-63fee82e-f507-4b05-bf66-d9e402d1f79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:19.173929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5772dc-6813-4b83-ac77-174b905ede08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:19.174859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5772dc-6813-4b83-ac77-174b905ede08 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:19.175313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5772dc-6813-4b83-ac77-174b905ede08 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:19.175777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5772dc-6813-4b83-ac77-174b905ede08 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:19.177535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5772dc-6813-4b83-ac77-174b905ede08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:19.179587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5772dc-6813-4b83-ac77-174b905ede08 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:19.181946 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 58/118] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:19.182817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5772dc-6813-4b83-ac77-174b905ede08 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:19.262992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 61/119] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:19.264401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fee82e-f507-4b05-bf66-d9e402d1f79e None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:19.399017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5772dc-6813-4b83-ac77-174b905ede08 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:19.404338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fee82e-f507-4b05-bf66-d9e402d1f79e None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:19.406122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:19.435014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c56f92-8b73-4a4c-93c5-bfffb441acee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:19.435836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c56f92-8b73-4a4c-93c5-bfffb441acee None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:19.436210 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c56f92-8b73-4a4c-93c5-bfffb441acee None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:19.436679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c56f92-8b73-4a4c-93c5-bfffb441acee None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360/users/34554fb083404728bd4ff4fa2b4a8149/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:19.438881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c56f92-8b73-4a4c-93c5-bfffb441acee None admin] RBAC: Authorizing `identity:create_grant(project_id=07768f903a224486a253fc7acc9c3360, user_id=34554fb083404728bd4ff4fa2b4a8149, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:19.441392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c56f92-8b73-4a4c-93c5-bfffb441acee None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:19.443596 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577d203-31bf-4fe0-85b3-3f22e9f7dd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:19.444629 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577d203-31bf-4fe0-85b3-3f22e9f7dd6a None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:19.445636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577d203-31bf-4fe0-85b3-3f22e9f7dd6a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:19.446083 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577d203-31bf-4fe0-85b3-3f22e9f7dd6a None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:19.447048 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577d203-31bf-4fe0-85b3-3f22e9f7dd6a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:19.448755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0577d203-31bf-4fe0-85b3-3f22e9f7dd6a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:19.553566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 59/120] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:33:19 2022] PUT /identity/v3/projects/07768f903a224486a253fc7acc9c3360/users/34554fb083404728bd4ff4fa2b4a8149/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:19.554730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c56f92-8b73-4a4c-93c5-bfffb441acee None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:19.577550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:19.580359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 62/121] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 19:33:19 2022] POST /identity/v3/projects => generated 302 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:33:19.581323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577d203-31bf-4fe0-85b3-3f22e9f7dd6a None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:19.729540 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577d203-31bf-4fe0-85b3-3f22e9f7dd6a None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:19.765703 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c56f92-8b73-4a4c-93c5-bfffb441acee None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:19.789207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a9568-ca10-4c7a-8780-ccbb71f64f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:19.790306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a9568-ca10-4c7a-8780-ccbb71f64f33 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:19.790805 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a9568-ca10-4c7a-8780-ccbb71f64f33 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:19.791332 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a9568-ca10-4c7a-8780-ccbb71f64f33 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:19.803953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a9568-ca10-4c7a-8780-ccbb71f64f33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-6e4a9568-ca10-4c7a-8780-ccbb71f64f33 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:19.810189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:19.811098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:19.812466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:19.812466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:19.812466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:19.812466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:19.812466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:19.812466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:19.812466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:19.812466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:19.812466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:19.812466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:19.812466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:19.812466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:19.814964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 63/122] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:33:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:19.815387 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a9568-ca10-4c7a-8780-ccbb71f64f33 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:19.818745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb51bf7-afaf-47ed-bb4c-b58690895e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:19.819744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb51bf7-afaf-47ed-bb4c-b58690895e17 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:19.820247 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb51bf7-afaf-47ed-bb4c-b58690895e17 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:19.820714 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb51bf7-afaf-47ed-bb4c-b58690895e17 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:19.838111 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb51bf7-afaf-47ed-bb4c-b58690895e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:19.839207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-cfb51bf7-afaf-47ed-bb4c-b58690895e17 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:19.847805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:19.850727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:19.851500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:19.851500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:19.851500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:19.851500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:19.851500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:19.851500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:19.851500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:19.851500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:19.851500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:19.851500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:19.851500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:19.851500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:19.852760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 60/123] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:19.858821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb51bf7-afaf-47ed-bb4c-b58690895e17 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:19.964045 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:19.989274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a9568-ca10-4c7a-8780-ccbb71f64f33 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:20.001143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb51bf7-afaf-47ed-bb4c-b58690895e17 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:20.035475 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51242a0-5107-460e-bfb9-14474d36243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:20.036250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51242a0-5107-460e-bfb9-14474d36243e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:20.036624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51242a0-5107-460e-bfb9-14474d36243e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:20.036904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51242a0-5107-460e-bfb9-14474d36243e None admin] PATH_INFO: `/v3/roles/member` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:20.038640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804affb2-2b00-4620-8412-d3b569336257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:20.039465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804affb2-2b00-4620-8412-d3b569336257 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:20.039965 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804affb2-2b00-4620-8412-d3b569336257 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:20.040333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804affb2-2b00-4620-8412-d3b569336257 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:20.041113 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804affb2-2b00-4620-8412-d3b569336257 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:20.041509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804affb2-2b00-4620-8412-d3b569336257 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:20.043036 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804affb2-2b00-4620-8412-d3b569336257 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:20.050718 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51242a0-5107-460e-bfb9-14474d36243e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-c51242a0-5107-460e-bfb9-14474d36243e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:20.056064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:20.057054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:20.058146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:20.058146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:20.058146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:20.058146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:20.058146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:20.058146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:20.058146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:20.058146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:20.058146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:20.058146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:20.058146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:20.058146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:20.058924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 61/124] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:33:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:20.058924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804affb2-2b00-4620-8412-d3b569336257 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:20.060284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 64/125] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:20.067284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51242a0-5107-460e-bfb9-14474d36243e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:20.180449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804affb2-2b00-4620-8412-d3b569336257 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:20.226802 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:20.232011 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe02d60-7375-44ea-aa65-9bbb9b8521e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:20.232640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe02d60-7375-44ea-aa65-9bbb9b8521e0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:20.232926 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe02d60-7375-44ea-aa65-9bbb9b8521e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:20.233752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe02d60-7375-44ea-aa65-9bbb9b8521e0 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:20.234444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe02d60-7375-44ea-aa65-9bbb9b8521e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:20.240041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe02d60-7375-44ea-aa65-9bbb9b8521e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:20.240041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe02d60-7375-44ea-aa65-9bbb9b8521e0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:20.240601 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:20.257435 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 62/126] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:20.258318 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe02d60-7375-44ea-aa65-9bbb9b8521e0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:20.279770 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51242a0-5107-460e-bfb9-14474d36243e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:20.335334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057657e0-0c6b-4ada-be4e-360fbfc21396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:20.335899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057657e0-0c6b-4ada-be4e-360fbfc21396 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:20.336191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057657e0-0c6b-4ada-be4e-360fbfc21396 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:20.336471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057657e0-0c6b-4ada-be4e-360fbfc21396 None admin] PATH_INFO: `/v3/users/34554fb083404728bd4ff4fa2b4a8149` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:20.337741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057657e0-0c6b-4ada-be4e-360fbfc21396 None admin] RBAC: Authorizing `identity:get_user(user_id=34554fb083404728bd4ff4fa2b4a8149)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:20.339245 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057657e0-0c6b-4ada-be4e-360fbfc21396 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:20.341169 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 65/127] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:20 2022] GET /identity/v3/users/34554fb083404728bd4ff4fa2b4a8149 => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:20.341999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057657e0-0c6b-4ada-be4e-360fbfc21396 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:20.460867 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe02d60-7375-44ea-aa65-9bbb9b8521e0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:20.508520 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727c2cf-214d-4a05-8f2f-c550fee9fe99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:20.509232 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727c2cf-214d-4a05-8f2f-c550fee9fe99 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:20.509587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727c2cf-214d-4a05-8f2f-c550fee9fe99 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:20.509973 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727c2cf-214d-4a05-8f2f-c550fee9fe99 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:20.510699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727c2cf-214d-4a05-8f2f-c550fee9fe99 None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:20.511073 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727c2cf-214d-4a05-8f2f-c550fee9fe99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:20.512717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727c2cf-214d-4a05-8f2f-c550fee9fe99 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:20.526041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 63/128] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:20.526999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727c2cf-214d-4a05-8f2f-c550fee9fe99 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:20.547841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057657e0-0c6b-4ada-be4e-360fbfc21396 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:20.610918 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:20.612033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b079a9-1a1f-4086-86de-e7a1cf0814dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:20.612673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b079a9-1a1f-4086-86de-e7a1cf0814dd None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:20.612954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b079a9-1a1f-4086-86de-e7a1cf0814dd None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:20.613222 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b079a9-1a1f-4086-86de-e7a1cf0814dd None admin] PATH_INFO: `/v3/users/043f31f4faa940b7a300612a90db95ba` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:20.645426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b079a9-1a1f-4086-86de-e7a1cf0814dd None admin] RBAC: Authorizing `identity:get_user(user_id=043f31f4faa940b7a300612a90db95ba)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:20.646985 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b079a9-1a1f-4086-86de-e7a1cf0814dd None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:20.648751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 66/129] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:20 2022] GET /identity/v3/users/043f31f4faa940b7a300612a90db95ba => generated 289 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:20.649584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b079a9-1a1f-4086-86de-e7a1cf0814dd None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:20.662161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:20.688059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727c2cf-214d-4a05-8f2f-c550fee9fe99 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:20.726356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f640290-c379-457b-8c9e-2c80d3fa4044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:20.727358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f640290-c379-457b-8c9e-2c80d3fa4044 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:20.727652 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f640290-c379-457b-8c9e-2c80d3fa4044 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:20.727934 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f640290-c379-457b-8c9e-2c80d3fa4044 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:20.729405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f640290-c379-457b-8c9e-2c80d3fa4044 None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:20.730895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f640290-c379-457b-8c9e-2c80d3fa4044 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:20.732527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 64/130] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:20 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:20.733463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f640290-c379-457b-8c9e-2c80d3fa4044 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:20.837612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f640290-c379-457b-8c9e-2c80d3fa4044 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:20.888414 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:20.895413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697461ad-3550-4261-b64b-ca9e26541ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:20.896035 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697461ad-3550-4261-b64b-ca9e26541ffa None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:20.896316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697461ad-3550-4261-b64b-ca9e26541ffa None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:20.896659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697461ad-3550-4261-b64b-ca9e26541ffa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:20.908913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b079a9-1a1f-4086-86de-e7a1cf0814dd None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:20.972698 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697461ad-3550-4261-b64b-ca9e26541ffa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:20.974234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697461ad-3550-4261-b64b-ca9e26541ffa None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:21.023696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb781276-bfda-4afa-8534-6a8b588b393c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:21.025765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb781276-bfda-4afa-8534-6a8b588b393c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:21.026342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb781276-bfda-4afa-8534-6a8b588b393c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:21.027300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb781276-bfda-4afa-8534-6a8b588b393c None admin] PATH_INFO: `/v3/users/34554fb083404728bd4ff4fa2b4a8149` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:21.029633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb781276-bfda-4afa-8534-6a8b588b393c None admin] RBAC: Authorizing `identity:get_user(user_id=34554fb083404728bd4ff4fa2b4a8149)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:21.034133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb781276-bfda-4afa-8534-6a8b588b393c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:21.036226 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 67/131] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:20 2022] GET /identity/v3/users/34554fb083404728bd4ff4fa2b4a8149 => generated 275 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:21.039223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb781276-bfda-4afa-8534-6a8b588b393c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:21.103292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 65/132] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:21.104830 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697461ad-3550-4261-b64b-ca9e26541ffa None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:21.146575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:21.191018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:21.225673 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697461ad-3550-4261-b64b-ca9e26541ffa None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:21.265321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903de60-db6a-4ff6-9039-f84b4599f0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:21.266478 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903de60-db6a-4ff6-9039-f84b4599f0b7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:21.267006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903de60-db6a-4ff6-9039-f84b4599f0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:21.267681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903de60-db6a-4ff6-9039-f84b4599f0b7 None admin] PATH_INFO: `/v3/projects/eff97a5d26c54873ae5acd2f1d454e39` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:21.269582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903de60-db6a-4ff6-9039-f84b4599f0b7 None admin] RBAC: Authorizing `identity:get_project(project_id=eff97a5d26c54873ae5acd2f1d454e39)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:21.271310 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9903de60-db6a-4ff6-9039-f84b4599f0b7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:21.273215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 66/133] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:21 2022] GET /identity/v3/projects/eff97a5d26c54873ae5acd2f1d454e39 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:21.274381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903de60-db6a-4ff6-9039-f84b4599f0b7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:21.285537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb781276-bfda-4afa-8534-6a8b588b393c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:21.357338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb2645-a957-4a8c-9e67-a57053b54f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:21.358035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb2645-a957-4a8c-9e67-a57053b54f5f None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:21.358381 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb2645-a957-4a8c-9e67-a57053b54f5f None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:21.358659 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb2645-a957-4a8c-9e67-a57053b54f5f None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360/users/34554fb083404728bd4ff4fa2b4a8149/roles/36b107b4892f47f8af6b65e1d1d94bb0` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:21.363057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb2645-a957-4a8c-9e67-a57053b54f5f None admin] RBAC: Authorizing `identity:create_grant(project_id=07768f903a224486a253fc7acc9c3360, user_id=34554fb083404728bd4ff4fa2b4a8149, role_id=36b107b4892f47f8af6b65e1d1d94bb0)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:21.364533 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbb2645-a957-4a8c-9e67-a57053b54f5f None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:21.413490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:21.421276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903de60-db6a-4ff6-9039-f84b4599f0b7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:21.457792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b761c9-a665-4368-81c6-6e008239791a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:21.458447 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 68/134] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:33:21 2022] PUT /identity/v3/projects/07768f903a224486a253fc7acc9c3360/users/34554fb083404728bd4ff4fa2b4a8149/roles/36b107b4892f47f8af6b65e1d1d94bb0 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:21.458929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb2645-a957-4a8c-9e67-a57053b54f5f None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:21.459525 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b761c9-a665-4368-81c6-6e008239791a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:21.460001 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b761c9-a665-4368-81c6-6e008239791a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:21.460480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b761c9-a665-4368-81c6-6e008239791a None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:21.462044 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b761c9-a665-4368-81c6-6e008239791a None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:21.464474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b761c9-a665-4368-81c6-6e008239791a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:21.466683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 67/135] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:21 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:21.469005 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ca1d2-5dbf-4010-8a0f-fdf75f2931ce None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:21.469496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ca1d2-5dbf-4010-8a0f-fdf75f2931ce None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:21.470014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ca1d2-5dbf-4010-8a0f-fdf75f2931ce None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:21.471879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 68/136] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:21.478773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ca1d2-5dbf-4010-8a0f-fdf75f2931ce None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:21.569664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbb2645-a957-4a8c-9e67-a57053b54f5f None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:21.602166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ca1d2-5dbf-4010-8a0f-fdf75f2931ce None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:21.615262 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:21.634633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3191b6-7778-48fb-8a20-5da46c45c226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:21.635392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3191b6-7778-48fb-8a20-5da46c45c226 None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:21.635613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3191b6-7778-48fb-8a20-5da46c45c226 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:21.635895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3191b6-7778-48fb-8a20-5da46c45c226 None admin] PATH_INFO: `/v3/projects/eff97a5d26c54873ae5acd2f1d454e39/users/34554fb083404728bd4ff4fa2b4a8149/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:21.638006 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3191b6-7778-48fb-8a20-5da46c45c226 None admin] RBAC: Authorizing `identity:create_grant(project_id=eff97a5d26c54873ae5acd2f1d454e39, user_id=34554fb083404728bd4ff4fa2b4a8149, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:21.639553 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3191b6-7778-48fb-8a20-5da46c45c226 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:21.647995 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9149b4-6d40-4456-8373-837ab9e4d30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:21.649055 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9149b4-6d40-4456-8373-837ab9e4d30a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:21.649558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9149b4-6d40-4456-8373-837ab9e4d30a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:21.650124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9149b4-6d40-4456-8373-837ab9e4d30a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:21.651122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9149b4-6d40-4456-8373-837ab9e4d30a None admin] RBAC: Adding query filter params (role.id=35aa74738f2d4844af99831724f20d39, scope.project.id=07768f903a224486a253fc7acc9c3360, user.id=043f31f4faa940b7a300612a90db95ba) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:21.651695 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9149b4-6d40-4456-8373-837ab9e4d30a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:21.653586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9149b4-6d40-4456-8373-837ab9e4d30a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:21.661135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 69/137] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:21 2022] GET /identity/v3/role_assignments?user.id=043f31f4faa940b7a300612a90db95ba&scope.project.id=07768f903a224486a253fc7acc9c3360&role.id=35aa74738f2d4844af99831724f20d39 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:21.662779 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194a18c-ee6c-47e4-ab2d-51756ebb7ac4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:21.663414 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194a18c-ee6c-47e4-ab2d-51756ebb7ac4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:21.664024 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6194a18c-ee6c-47e4-ab2d-51756ebb7ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:21.685943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6194a18c-ee6c-47e4-ab2d-51756ebb7ac4 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:21.688235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6194a18c-ee6c-47e4-ab2d-51756ebb7ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:21.713505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:21.720521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 69/138] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:33:21 2022] PUT /identity/v3/projects/eff97a5d26c54873ae5acd2f1d454e39/users/34554fb083404728bd4ff4fa2b4a8149/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:21.769390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 70/139] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:22.155506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f876637-e587-4caa-b276-5e5807ce1412 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:22.155869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f876637-e587-4caa-b276-5e5807ce1412 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:22.156078 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f876637-e587-4caa-b276-5e5807ce1412 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:22.157743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 70/140] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:22.163181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42040c5-775b-47fe-8133-03f4aef5e9c2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:22.163472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42040c5-775b-47fe-8133-03f4aef5e9c2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:22.163756 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42040c5-775b-47fe-8133-03f4aef5e9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:22.182488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f42040c5-775b-47fe-8133-03f4aef5e9c2 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:22.183683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f42040c5-775b-47fe-8133-03f4aef5e9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:22.263654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 71/141] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:22.374951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f876637-e587-4caa-b276-5e5807ce1412 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:22.504790 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f876637-e587-4caa-b276-5e5807ce1412 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:22.554872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a99511-e05b-4c66-915c-16f5c6e3b7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:22.555525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a99511-e05b-4c66-915c-16f5c6e3b7f6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:22.555806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a99511-e05b-4c66-915c-16f5c6e3b7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:22.556078 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a99511-e05b-4c66-915c-16f5c6e3b7f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:22.567963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a99511-e05b-4c66-915c-16f5c6e3b7f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-e7a99511-e05b-4c66-915c-16f5c6e3b7f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:22.572791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:22.573705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:22.574492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:22.574492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:22.574492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:22.574492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:22.574492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:22.574492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:22.574492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:22.574492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:22.574492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:22.574492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:22.574492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:22.574492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:22.574492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 71/142] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:33:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:33:22.581719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42040c5-775b-47fe-8133-03f4aef5e9c2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:22.687853 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a99511-e05b-4c66-915c-16f5c6e3b7f6 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:22.690793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:22.716371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42040c5-775b-47fe-8133-03f4aef5e9c2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:22.748045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:22.782505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06c175-b9a4-4974-a383-58fb969bd398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:22.783472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06c175-b9a4-4974-a383-58fb969bd398 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:22.783938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06c175-b9a4-4974-a383-58fb969bd398 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:22.784394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06c175-b9a4-4974-a383-58fb969bd398 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:22.785822 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06c175-b9a4-4974-a383-58fb969bd398 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:22.786562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06c175-b9a4-4974-a383-58fb969bd398 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:22.788630 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06c175-b9a4-4974-a383-58fb969bd398 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:22.803434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 72/143] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:22.803434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ff5b3-aae4-4029-b2fc-046e610fb660 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:22.803434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ff5b3-aae4-4029-b2fc-046e610fb660 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:22.803434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ff5b3-aae4-4029-b2fc-046e610fb660 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:22.803434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 73/144] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:22.811164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ff5b3-aae4-4029-b2fc-046e610fb660 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:22.829952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a99511-e05b-4c66-915c-16f5c6e3b7f6 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:22.901062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8496b0a-760b-4033-9856-ff675811b3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:22.901703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8496b0a-760b-4033-9856-ff675811b3fc None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:22.901990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8496b0a-760b-4033-9856-ff675811b3fc None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:22.902260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8496b0a-760b-4033-9856-ff675811b3fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:22.918760 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8496b0a-760b-4033-9856-ff675811b3fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-b8496b0a-760b-4033-9856-ff675811b3fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:22.923584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:22.924423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:22.925205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:22.925205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:22.925205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:22.925205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:22.925205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:22.925205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:22.925205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:22.925205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:22.925205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:22.925205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:22.925205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:22.925205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:22.926277 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 72/145] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:33:22.927170 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b288d-fcd8-44c3-a997-95617fb00c17 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:22.927468 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b288d-fcd8-44c3-a997-95617fb00c17 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:22.927739 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b288d-fcd8-44c3-a997-95617fb00c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:22.960844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-ac1b288d-fcd8-44c3-a997-95617fb00c17 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:22.962060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ac1b288d-fcd8-44c3-a997-95617fb00c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:22.975549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ff5b3-aae4-4029-b2fc-046e610fb660 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:23.042344 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44d631-843b-4e97-9120-d3906f500e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:23.042976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44d631-843b-4e97-9120-d3906f500e24 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:23.043254 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44d631-843b-4e97-9120-d3906f500e24 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:23.043526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44d631-843b-4e97-9120-d3906f500e24 None admin] PATH_INFO: `/v3/users/e8f45104d2d540658c4bbc32613a9317` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:23.045371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44d631-843b-4e97-9120-d3906f500e24 None admin] RBAC: Authorizing `identity:get_user(user_id=e8f45104d2d540658c4bbc32613a9317)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:23.046986 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44d631-843b-4e97-9120-d3906f500e24 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:23.053978 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 74/146] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:22 2022] GET /identity/v3/users/e8f45104d2d540658c4bbc32613a9317 => generated 247 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:23.054859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44d631-843b-4e97-9120-d3906f500e24 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:23.080699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:23.127082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 73/147] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:23.127944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b288d-fcd8-44c3-a997-95617fb00c17 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:23.182032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44d631-843b-4e97-9120-d3906f500e24 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:23.215748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:23.239444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3ea25-4e31-47c8-a715-15258aece260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:23.240066 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3ea25-4e31-47c8-a715-15258aece260 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:23.240347 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3ea25-4e31-47c8-a715-15258aece260 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:23.240618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f3ea25-4e31-47c8-a715-15258aece260 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:23.241272 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3ea25-4e31-47c8-a715-15258aece260 None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:23.241574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3ea25-4e31-47c8-a715-15258aece260 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:23.243001 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f3ea25-4e31-47c8-a715-15258aece260 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:23.257239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 75/148] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:23.268265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3ea25-4e31-47c8-a715-15258aece260 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:23.303136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b288d-fcd8-44c3-a997-95617fb00c17 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:23.356043 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99d578-d33b-47f5-b9e3-465d238359bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:23.356943 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99d578-d33b-47f5-b9e3-465d238359bf None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:23.357392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99d578-d33b-47f5-b9e3-465d238359bf None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:23.357858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c99d578-d33b-47f5-b9e3-465d238359bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:23.447945 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99d578-d33b-47f5-b9e3-465d238359bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:23.450342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c99d578-d33b-47f5-b9e3-465d238359bf None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:23.495711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:23.521269 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:23.548812 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f3ea25-4e31-47c8-a715-15258aece260 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:23.585059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23459c7e-541f-4ddc-b486-1f5994911833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:23.585654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23459c7e-541f-4ddc-b486-1f5994911833 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:23.585915 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23459c7e-541f-4ddc-b486-1f5994911833 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:23.586193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23459c7e-541f-4ddc-b486-1f5994911833 None admin] PATH_INFO: `/v3/users/34554fb083404728bd4ff4fa2b4a8149` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:23.587339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23459c7e-541f-4ddc-b486-1f5994911833 None admin] RBAC: Authorizing `identity:get_user(user_id=34554fb083404728bd4ff4fa2b4a8149)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:23.588895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23459c7e-541f-4ddc-b486-1f5994911833 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:23.590754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 76/149] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:23 2022] GET /identity/v3/users/34554fb083404728bd4ff4fa2b4a8149 => generated 275 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:23.591434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23459c7e-541f-4ddc-b486-1f5994911833 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:23.630530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 74/150] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:23.632211 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99d578-d33b-47f5-b9e3-465d238359bf None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:23.757182 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23459c7e-541f-4ddc-b486-1f5994911833 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:23.810904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c99d578-d33b-47f5-b9e3-465d238359bf None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:23.823102 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24b34d-b320-4946-a744-64326be512ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:23.824053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24b34d-b320-4946-a744-64326be512ab None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:23.824522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24b34d-b320-4946-a744-64326be512ab None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:23.824978 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24b34d-b320-4946-a744-64326be512ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:23.826293 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24b34d-b320-4946-a744-64326be512ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:23.828779 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24b34d-b320-4946-a744-64326be512ab None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:23.832178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 77/151] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:23.833334 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24b34d-b320-4946-a744-64326be512ab None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:23.871149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286270f-1a71-44f5-9862-6b2d8ca438e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:23.872081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286270f-1a71-44f5-9862-6b2d8ca438e0 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:23.872641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286270f-1a71-44f5-9862-6b2d8ca438e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:23.874445 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:23.877976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286270f-1a71-44f5-9862-6b2d8ca438e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:23.900502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:23.975025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24b34d-b320-4946-a744-64326be512ab None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:24.009089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4e592-bab9-42c9-9b7a-24bff306d152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:24.009716 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4e592-bab9-42c9-9b7a-24bff306d152 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:24.010023 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4e592-bab9-42c9-9b7a-24bff306d152 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:24.010297 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4e592-bab9-42c9-9b7a-24bff306d152 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:24.011421 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4e592-bab9-42c9-9b7a-24bff306d152 None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:24.012855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4e592-bab9-42c9-9b7a-24bff306d152 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:24.015084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 78/152] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:23 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:24.015826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4e592-bab9-42c9-9b7a-24bff306d152 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:24.017441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286270f-1a71-44f5-9862-6b2d8ca438e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:24.020180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286270f-1a71-44f5-9862-6b2d8ca438e0 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:24.154341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 75/153] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:24.155298 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286270f-1a71-44f5-9862-6b2d8ca438e0 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:24.158165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:24.177849 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4e592-bab9-42c9-9b7a-24bff306d152 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:24.198921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:24.216265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505afe3-93c6-4794-968f-0bf623c0a429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:24.217241 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505afe3-93c6-4794-968f-0bf623c0a429 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:24.218372 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505afe3-93c6-4794-968f-0bf623c0a429 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:24.218701 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2505afe3-93c6-4794-968f-0bf623c0a429 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:24.219430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505afe3-93c6-4794-968f-0bf623c0a429 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:24.220916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2505afe3-93c6-4794-968f-0bf623c0a429 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:24.265993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286270f-1a71-44f5-9862-6b2d8ca438e0 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:24.301934 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1aec5f-1927-4b3c-ad01-d9ba0632ac3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:24.302868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1aec5f-1927-4b3c-ad01-d9ba0632ac3b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:24.303322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1aec5f-1927-4b3c-ad01-d9ba0632ac3b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:24.303816 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1aec5f-1927-4b3c-ad01-d9ba0632ac3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:24.304720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1aec5f-1927-4b3c-ad01-d9ba0632ac3b None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=07768f903a224486a253fc7acc9c3360, user.id=e8f45104d2d540658c4bbc32613a9317) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:24.305213 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1aec5f-1927-4b3c-ad01-d9ba0632ac3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:24.307090 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1aec5f-1927-4b3c-ad01-d9ba0632ac3b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:24.314451 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 76/154] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:24 2022] GET /identity/v3/role_assignments?user.id=e8f45104d2d540658c4bbc32613a9317&scope.project.id=07768f903a224486a253fc7acc9c3360&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:24.315526 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1aec5f-1927-4b3c-ad01-d9ba0632ac3b None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:24.341302 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 79/155] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:33:24 2022] POST /identity/v3/users => generated 283 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:33:24.448976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1aec5f-1927-4b3c-ad01-d9ba0632ac3b None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:24.451685 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:24.501294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5d5ca1-bb90-4190-bc86-f608552a1c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:24.502007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d5ca1-bb90-4190-bc86-f608552a1c8b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:24.502288 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d5ca1-bb90-4190-bc86-f608552a1c8b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:24.502563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5d5ca1-bb90-4190-bc86-f608552a1c8b None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:24.503796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d5ca1-bb90-4190-bc86-f608552a1c8b None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:24.505253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5d5ca1-bb90-4190-bc86-f608552a1c8b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:24.506984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 77/156] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:24 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:24.513172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505afe3-93c6-4794-968f-0bf623c0a429 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:24.543054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57747080-9775-4c32-8d5d-b361a045af31 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:24.543054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57747080-9775-4c32-8d5d-b361a045af31 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:24.543054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57747080-9775-4c32-8d5d-b361a045af31 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:24.544137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 78/157] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:33:24.556541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcb6e9-d0d5-414d-a8a5-29c79a2b5b76 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:24.556834 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcb6e9-d0d5-414d-a8a5-29c79a2b5b76 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:24.557099 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfcb6e9-d0d5-414d-a8a5-29c79a2b5b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:24.603120 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-4cfcb6e9-d0d5-414d-a8a5-29c79a2b5b76 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:24.604560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4cfcb6e9-d0d5-414d-a8a5-29c79a2b5b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:24.641298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2505afe3-93c6-4794-968f-0bf623c0a429 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:24.698632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89527f24-506a-4439-8c81-7f87ec26ae4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:24.699323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89527f24-506a-4439-8c81-7f87ec26ae4b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:24.699543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89527f24-506a-4439-8c81-7f87ec26ae4b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:24.699813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89527f24-506a-4439-8c81-7f87ec26ae4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:24.700548 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89527f24-506a-4439-8c81-7f87ec26ae4b None admin] RBAC: Adding query filter params (role.id=efa4ca02b2ff4ad6b14d501d4d79bbea, scope.project.id=07768f903a224486a253fc7acc9c3360, user.id=34554fb083404728bd4ff4fa2b4a8149) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:24.700852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89527f24-506a-4439-8c81-7f87ec26ae4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:24.702913 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89527f24-506a-4439-8c81-7f87ec26ae4b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:24.714654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 80/158] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:24 2022] GET /identity/v3/role_assignments?user.id=34554fb083404728bd4ff4fa2b4a8149&scope.project.id=07768f903a224486a253fc7acc9c3360&role.id=efa4ca02b2ff4ad6b14d501d4d79bbea => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:24.738993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 79/159] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 14 19:33:24.813048 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6688dcf-dec5-449c-8ff1-d0ce8d35c642 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:24.813361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6688dcf-dec5-449c-8ff1-d0ce8d35c642 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:24.813600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6688dcf-dec5-449c-8ff1-d0ce8d35c642 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:24.814968 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 81/160] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:24.818995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88e539-d955-4e4a-b070-263ceb531b8a None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:24.819287 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88e539-d955-4e4a-b070-263ceb531b8a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:24.819577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88e539-d955-4e4a-b070-263ceb531b8a None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:24.821366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 80/161] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:24.822090 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c902e-3a3e-4814-bb4b-db9d45b1be31 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:24.822577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c902e-3a3e-4814-bb4b-db9d45b1be31 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:24.823045 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c902e-3a3e-4814-bb4b-db9d45b1be31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:24.827662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6981522-2c4c-4a66-ae2d-3e80a38e208c None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:24.827956 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6981522-2c4c-4a66-ae2d-3e80a38e208c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:24.828225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6981522-2c4c-4a66-ae2d-3e80a38e208c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:24.832193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:24.845613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-131c902e-3a3e-4814-bb4b-db9d45b1be31 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:24.849183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-131c902e-3a3e-4814-bb4b-db9d45b1be31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:24.850426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e6981522-2c4c-4a66-ae2d-3e80a38e208c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:24.851845 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e6981522-2c4c-4a66-ae2d-3e80a38e208c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:24.949923 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 82/162] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:24.957994 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 81/163] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:25.059553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c902e-3a3e-4814-bb4b-db9d45b1be31 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:25.168748 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c902e-3a3e-4814-bb4b-db9d45b1be31 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:25.200697 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:25.206299 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7307a2-618d-46f9-968a-b18b558b4729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:25.214475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7307a2-618d-46f9-968a-b18b558b4729 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:25.214475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7307a2-618d-46f9-968a-b18b558b4729 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:25.214475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7307a2-618d-46f9-968a-b18b558b4729 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:25.219245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7307a2-618d-46f9-968a-b18b558b4729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-bb7307a2-618d-46f9-968a-b18b558b4729 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:25.224391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:25.225469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:25.226447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:25.226447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:25.226447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:25.226447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:25.226447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:25.226447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:25.226447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:25.226447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:25.226447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:25.226447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:25.226447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:25.226447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:25.228625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 83/164] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:25.234429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6981522-2c4c-4a66-ae2d-3e80a38e208c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:25.244797 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7307a2-618d-46f9-968a-b18b558b4729 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:25.278646 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:25.366231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6981522-2c4c-4a66-ae2d-3e80a38e208c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:25.381401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7307a2-618d-46f9-968a-b18b558b4729 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:25.413130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3e341-7c97-468f-9045-56b97bf76bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:25.413875 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3e341-7c97-468f-9045-56b97bf76bbe None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:25.414185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3e341-7c97-468f-9045-56b97bf76bbe None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:25.414554 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3e341-7c97-468f-9045-56b97bf76bbe None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:25.415217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3e341-7c97-468f-9045-56b97bf76bbe None admin] RBAC: Adding query filter params (name=reader) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:25.415519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3e341-7c97-468f-9045-56b97bf76bbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:25.417010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3e341-7c97-468f-9045-56b97bf76bbe None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:25.429147 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 82/165] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:25.430760 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3e341-7c97-468f-9045-56b97bf76bbe None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:25.436805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098d4f4-429f-4656-bde6-497cbd60a079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:25.438617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098d4f4-429f-4656-bde6-497cbd60a079 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:25.438987 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098d4f4-429f-4656-bde6-497cbd60a079 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:25.439331 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098d4f4-429f-4656-bde6-497cbd60a079 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:25.451145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6098d4f4-429f-4656-bde6-497cbd60a079 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-6098d4f4-429f-4656-bde6-497cbd60a079 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:25.456970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:25.457866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:25.459152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:25.459152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:25.459152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:25.459152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:25.459152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:25.459152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:25.459152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:25.459152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:25.459152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:25.459152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:25.459152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:25.459152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:25.459152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 84/166] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:33:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:33:25.459806 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098d4f4-429f-4656-bde6-497cbd60a079 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:25.528503 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:25.566589 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:25.578352 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3e341-7c97-468f-9045-56b97bf76bbe None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:25.591283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098d4f4-429f-4656-bde6-497cbd60a079 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:25.618749 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd10bf-6043-40a2-8daa-867ea3eaedbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:25.619234 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd10bf-6043-40a2-8daa-867ea3eaedbf None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:25.619513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd10bf-6043-40a2-8daa-867ea3eaedbf None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:25.619785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd10bf-6043-40a2-8daa-867ea3eaedbf None admin] PATH_INFO: `/v3/roles/member` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:25.631817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd10bf-6043-40a2-8daa-867ea3eaedbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:25.639969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2003608-0f55-46ff-b9b4-d8dd5dd26042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-69dd10bf-6043-40a2-8daa-867ea3eaedbf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:25.640407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:25.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:25.642116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:25.642116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:25.642116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:25.642116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:25.642116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:25.642116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:25.642116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:25.642116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:25.642116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:25.642116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:25.642116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:25.642116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:25.642116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 83/167] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:25.642844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd10bf-6043-40a2-8daa-867ea3eaedbf None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:25.644218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2003608-0f55-46ff-b9b4-d8dd5dd26042 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:25.645414 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2003608-0f55-46ff-b9b4-d8dd5dd26042 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:25.645917 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2003608-0f55-46ff-b9b4-d8dd5dd26042 None admin] PATH_INFO: `/v3/users/043f31f4faa940b7a300612a90db95ba` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:25.648429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2003608-0f55-46ff-b9b4-d8dd5dd26042 None admin] RBAC: Authorizing `identity:get_user(user_id=043f31f4faa940b7a300612a90db95ba)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:25.650217 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2003608-0f55-46ff-b9b4-d8dd5dd26042 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:25.657652 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 85/168] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:25 2022] GET /identity/v3/users/043f31f4faa940b7a300612a90db95ba => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:25.659267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2003608-0f55-46ff-b9b4-d8dd5dd26042 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:25.755965 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd10bf-6043-40a2-8daa-867ea3eaedbf None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:25.781684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2003608-0f55-46ff-b9b4-d8dd5dd26042 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:25.792458 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaacb53-6c4e-4543-8484-79ba89f90496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:25.793166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaacb53-6c4e-4543-8484-79ba89f90496 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:25.793517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaacb53-6c4e-4543-8484-79ba89f90496 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:25.794115 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaacb53-6c4e-4543-8484-79ba89f90496 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:25.794625 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaacb53-6c4e-4543-8484-79ba89f90496 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:25.794997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaacb53-6c4e-4543-8484-79ba89f90496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:25.796511 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaacb53-6c4e-4543-8484-79ba89f90496 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:25.806332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:25.806476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 84/169] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:33:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:25.812431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaacb53-6c4e-4543-8484-79ba89f90496 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:25.826032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:25.841366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5bdc3-1d19-48d5-9822-0adcd1e3f0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:25.846887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5bdc3-1d19-48d5-9822-0adcd1e3f0c1 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:25.846887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5bdc3-1d19-48d5-9822-0adcd1e3f0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:25.846887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d5bdc3-1d19-48d5-9822-0adcd1e3f0c1 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:25.846887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5bdc3-1d19-48d5-9822-0adcd1e3f0c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:25.846887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5bdc3-1d19-48d5-9822-0adcd1e3f0c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:25.849976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d5bdc3-1d19-48d5-9822-0adcd1e3f0c1 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:25.871761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 86/170] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:25.872730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5bdc3-1d19-48d5-9822-0adcd1e3f0c1 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:25.981969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaacb53-6c4e-4543-8484-79ba89f90496 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:26.047095 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af58306-ae4c-4b1f-a190-2d298c073837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:26.047755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af58306-ae4c-4b1f-a190-2d298c073837 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:26.048034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af58306-ae4c-4b1f-a190-2d298c073837 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:26.048309 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af58306-ae4c-4b1f-a190-2d298c073837 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:26.049900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af58306-ae4c-4b1f-a190-2d298c073837 None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:26.051409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af58306-ae4c-4b1f-a190-2d298c073837 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:26.053381 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 85/171] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:25 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:26.058508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af58306-ae4c-4b1f-a190-2d298c073837 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:26.068188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d5bdc3-1d19-48d5-9822-0adcd1e3f0c1 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:26.130682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3907f58-cb13-4a41-ae20-238b9add7fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:26.131330 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3907f58-cb13-4a41-ae20-238b9add7fdf None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:26.131610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3907f58-cb13-4a41-ae20-238b9add7fdf None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:26.131932 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3907f58-cb13-4a41-ae20-238b9add7fdf None admin] PATH_INFO: `/v3/users/34554fb083404728bd4ff4fa2b4a8149` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:26.133138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3907f58-cb13-4a41-ae20-238b9add7fdf None admin] RBAC: Authorizing `identity:get_user(user_id=34554fb083404728bd4ff4fa2b4a8149)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:26.134635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3907f58-cb13-4a41-ae20-238b9add7fdf None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:26.141586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 87/172] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:25 2022] GET /identity/v3/users/34554fb083404728bd4ff4fa2b4a8149 => generated 275 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:26.142461 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3907f58-cb13-4a41-ae20-238b9add7fdf None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:26.224901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:26.233856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:26.256647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af58306-ae4c-4b1f-a190-2d298c073837 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:26.298721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3907f58-cb13-4a41-ae20-238b9add7fdf None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:26.332775 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658e846-bc33-4992-a2fe-aeb970f72d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:26.333419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658e846-bc33-4992-a2fe-aeb970f72d71 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:26.333695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658e846-bc33-4992-a2fe-aeb970f72d71 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:26.333989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658e846-bc33-4992-a2fe-aeb970f72d71 None admin] PATH_INFO: `/v3/users/34554fb083404728bd4ff4fa2b4a8149` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:26.335569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658e846-bc33-4992-a2fe-aeb970f72d71 None admin] RBAC: Authorizing `identity:get_user(user_id=34554fb083404728bd4ff4fa2b4a8149)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:26.337321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d658e846-bc33-4992-a2fe-aeb970f72d71 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:26.339634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 86/173] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:26 2022] GET /identity/v3/users/34554fb083404728bd4ff4fa2b4a8149 => generated 275 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:26.340228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658e846-bc33-4992-a2fe-aeb970f72d71 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:26.353990 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6a45c-7565-4b49-9315-5558ffa1ea55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:26.355027 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6a45c-7565-4b49-9315-5558ffa1ea55 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:26.355506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6a45c-7565-4b49-9315-5558ffa1ea55 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:26.355984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf6a45c-7565-4b49-9315-5558ffa1ea55 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360/users/043f31f4faa940b7a300612a90db95ba/roles/35aa74738f2d4844af99831724f20d39` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:26.358815 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6a45c-7565-4b49-9315-5558ffa1ea55 None admin] RBAC: Authorizing `identity:create_grant(project_id=07768f903a224486a253fc7acc9c3360, user_id=043f31f4faa940b7a300612a90db95ba, role_id=35aa74738f2d4844af99831724f20d39)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:26.360466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf6a45c-7565-4b49-9315-5558ffa1ea55 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:26.447241 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 88/174] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:33:26 2022] PUT /identity/v3/projects/07768f903a224486a253fc7acc9c3360/users/043f31f4faa940b7a300612a90db95ba/roles/35aa74738f2d4844af99831724f20d39 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:26.448639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6a45c-7565-4b49-9315-5558ffa1ea55 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:26.504658 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658e846-bc33-4992-a2fe-aeb970f72d71 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:26.551824 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48a6d8-6347-4280-83cd-cc25e361e19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:26.552431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48a6d8-6347-4280-83cd-cc25e361e19a None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:26.552719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48a6d8-6347-4280-83cd-cc25e361e19a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:26.552995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48a6d8-6347-4280-83cd-cc25e361e19a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:26.585936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:26.628703 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf6a45c-7565-4b49-9315-5558ffa1ea55 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:26.640121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:26.652197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48a6d8-6347-4280-83cd-cc25e361e19a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:26.653849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48a6d8-6347-4280-83cd-cc25e361e19a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:26.669924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858f177-7704-41a4-9e00-f0a40a43548a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:26.670694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f177-7704-41a4-9e00-f0a40a43548a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:26.670867 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f177-7704-41a4-9e00-f0a40a43548a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:26.671137 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858f177-7704-41a4-9e00-f0a40a43548a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:26.762725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858f177-7704-41a4-9e00-f0a40a43548a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:26.764195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858f177-7704-41a4-9e00-f0a40a43548a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:26.841340 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 87/175] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:26.847342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48a6d8-6347-4280-83cd-cc25e361e19a None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:26.868528 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 89/176] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:26.881829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858f177-7704-41a4-9e00-f0a40a43548a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:26.886705 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:26.938849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:26.977059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858f177-7704-41a4-9e00-f0a40a43548a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:27.011640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827114fa-6a54-4a10-b5eb-3cd07b5646ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:27.012268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827114fa-6a54-4a10-b5eb-3cd07b5646ba None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:27.012541 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827114fa-6a54-4a10-b5eb-3cd07b5646ba None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:27.012809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827114fa-6a54-4a10-b5eb-3cd07b5646ba None admin] PATH_INFO: `/v3/projects/eff97a5d26c54873ae5acd2f1d454e39` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:27.013948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827114fa-6a54-4a10-b5eb-3cd07b5646ba None admin] RBAC: Authorizing `identity:get_project(project_id=eff97a5d26c54873ae5acd2f1d454e39)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:27.015928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827114fa-6a54-4a10-b5eb-3cd07b5646ba None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:27.016579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48a6d8-6347-4280-83cd-cc25e361e19a None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:27.017855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 90/177] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:26 2022] GET /identity/v3/projects/eff97a5d26c54873ae5acd2f1d454e39 => generated 312 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:27.023474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827114fa-6a54-4a10-b5eb-3cd07b5646ba None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:27.053587 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f869b88-d1e6-4055-ba00-8facd109b85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:27.054263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f869b88-d1e6-4055-ba00-8facd109b85c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:27.054544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f869b88-d1e6-4055-ba00-8facd109b85c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:27.054821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f869b88-d1e6-4055-ba00-8facd109b85c None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:27.056024 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f869b88-d1e6-4055-ba00-8facd109b85c None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:27.057589 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f869b88-d1e6-4055-ba00-8facd109b85c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:27.064413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 88/178] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:26 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:27.065884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f869b88-d1e6-4055-ba00-8facd109b85c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:27.153040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:27.181972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f869b88-d1e6-4055-ba00-8facd109b85c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:27.183083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827114fa-6a54-4a10-b5eb-3cd07b5646ba None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:27.209995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:27.242736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209efd45-4dc9-4d69-a70e-136ed3d501d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:27.243451 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209efd45-4dc9-4d69-a70e-136ed3d501d5 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:27.243738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209efd45-4dc9-4d69-a70e-136ed3d501d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:27.244015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209efd45-4dc9-4d69-a70e-136ed3d501d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:27.244742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209efd45-4dc9-4d69-a70e-136ed3d501d5 None admin] RBAC: Adding query filter params (role.id=36b107b4892f47f8af6b65e1d1d94bb0, scope.project.id=07768f903a224486a253fc7acc9c3360, user.id=34554fb083404728bd4ff4fa2b4a8149) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:27.245053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209efd45-4dc9-4d69-a70e-136ed3d501d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:27.246470 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6691b5c-f422-40c5-ab92-adeda1a066ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:27.246591 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209efd45-4dc9-4d69-a70e-136ed3d501d5 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:27.247084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6691b5c-f422-40c5-ab92-adeda1a066ff None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:27.247364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6691b5c-f422-40c5-ab92-adeda1a066ff None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:27.247632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6691b5c-f422-40c5-ab92-adeda1a066ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:27.248345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6691b5c-f422-40c5-ab92-adeda1a066ff None admin] RBAC: Adding query filter params (role.id=efa4ca02b2ff4ad6b14d501d4d79bbea, scope.project.id=eff97a5d26c54873ae5acd2f1d454e39, user.id=34554fb083404728bd4ff4fa2b4a8149) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:27.249167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6691b5c-f422-40c5-ab92-adeda1a066ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:27.250606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6691b5c-f422-40c5-ab92-adeda1a066ff None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:27.256621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 91/179] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:27 2022] GET /identity/v3/role_assignments?user.id=34554fb083404728bd4ff4fa2b4a8149&scope.project.id=eff97a5d26c54873ae5acd2f1d454e39&role.id=efa4ca02b2ff4ad6b14d501d4d79bbea => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:27.259305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 89/180] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:27 2022] GET /identity/v3/role_assignments?user.id=34554fb083404728bd4ff4fa2b4a8149&scope.project.id=07768f903a224486a253fc7acc9c3360&role.id=36b107b4892f47f8af6b65e1d1d94bb0 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:27.852242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cceecbf-a3e5-4c43-86f7-8cd7b3633a82 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:27.852929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cceecbf-a3e5-4c43-86f7-8cd7b3633a82 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:27.852929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cceecbf-a3e5-4c43-86f7-8cd7b3633a82 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:27.856242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 92/181] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:27.860669 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b86eb-92ed-4fb5-8e70-beb90181acc6 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:27.861173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b86eb-92ed-4fb5-8e70-beb90181acc6 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:27.861324 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b86eb-92ed-4fb5-8e70-beb90181acc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:27.881327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-1d2b86eb-92ed-4fb5-8e70-beb90181acc6 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:27.882624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1d2b86eb-92ed-4fb5-8e70-beb90181acc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:27.960912 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 90/182] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:28.073557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543c4d9-d3f3-4295-87a9-ea7643dc031a None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:28.073998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543c4d9-d3f3-4295-87a9-ea7643dc031a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:28.074370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8543c4d9-d3f3-4295-87a9-ea7643dc031a None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:28.076129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 93/183] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:28.081954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a61f21-b810-4d74-a21b-c38e02638f62 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:28.082246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a61f21-b810-4d74-a21b-c38e02638f62 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:28.082526 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a61f21-b810-4d74-a21b-c38e02638f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:28.100782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-06a61f21-b810-4d74-a21b-c38e02638f62 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:28.102015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-06a61f21-b810-4d74-a21b-c38e02638f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:28.170792 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 91/184] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:28.247613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543c4d9-d3f3-4295-87a9-ea7643dc031a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:28.315992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccc536-2be4-4c6f-b9fd-6dbace2ff287 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:28.316743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccc536-2be4-4c6f-b9fd-6dbace2ff287 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:28.317501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ccc536-2be4-4c6f-b9fd-6dbace2ff287 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:28.319108 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 92/185] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:33:28.324527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28076520-7a06-417a-b648-92754865537f None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:28.325034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28076520-7a06-417a-b648-92754865537f None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:28.325511 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28076520-7a06-417a-b648-92754865537f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:28.345348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-28076520-7a06-417a-b648-92754865537f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:28.346912 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-28076520-7a06-417a-b648-92754865537f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:28.351732 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8543c4d9-d3f3-4295-87a9-ea7643dc031a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:28.387632 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:28.389611 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0c4b8-e4f5-4791-aa7d-d992f2c29b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:28.390439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0c4b8-e4f5-4791-aa7d-d992f2c29b6b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:28.390884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0c4b8-e4f5-4791-aa7d-d992f2c29b6b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:28.391296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0c4b8-e4f5-4791-aa7d-d992f2c29b6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:28.404769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0c4b8-e4f5-4791-aa7d-d992f2c29b6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-5cc0c4b8-e4f5-4791-aa7d-d992f2c29b6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:28.410267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:28.411162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:28.411948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:28.411948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:28.411948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:28.411948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:28.411948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:28.411948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:28.411948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:28.411948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:28.411948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:28.411948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:28.411948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:28.411948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:28.414797 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 94/186] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:33:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:28.421134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0c4b8-e4f5-4791-aa7d-d992f2c29b6b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:28.423373 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 93/187] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:28.424110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:28.471790 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28076520-7a06-417a-b648-92754865537f None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:28.528508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0c4b8-e4f5-4791-aa7d-d992f2c29b6b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:28.571081 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b356c7-6c1b-4223-9ac2-d8f0fdfb3881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:28.571948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b356c7-6c1b-4223-9ac2-d8f0fdfb3881 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:28.572361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b356c7-6c1b-4223-9ac2-d8f0fdfb3881 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:28.572770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b356c7-6c1b-4223-9ac2-d8f0fdfb3881 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:28.573581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b356c7-6c1b-4223-9ac2-d8f0fdfb3881 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:28.574043 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b356c7-6c1b-4223-9ac2-d8f0fdfb3881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:28.575616 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b356c7-6c1b-4223-9ac2-d8f0fdfb3881 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:28.586021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 95/188] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:28.591692 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b356c7-6c1b-4223-9ac2-d8f0fdfb3881 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:28.615678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28076520-7a06-417a-b648-92754865537f None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:28.662809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c90bdb-46db-4ebc-bc9c-8e694d46ed9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:28.663449 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c90bdb-46db-4ebc-bc9c-8e694d46ed9e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:28.663726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c90bdb-46db-4ebc-bc9c-8e694d46ed9e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:28.663997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c90bdb-46db-4ebc-bc9c-8e694d46ed9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:28.664940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:28.679092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c90bdb-46db-4ebc-bc9c-8e694d46ed9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-77c90bdb-46db-4ebc-bc9c-8e694d46ed9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:28.684136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:28.685050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:28.685867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:28.685867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:28.685867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:28.685867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:28.685867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:28.685867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:28.685867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:28.685867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:28.685867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:28.685867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:28.685867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:28.685867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:28.687952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 94/189] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:33:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:33:28.695380 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c90bdb-46db-4ebc-bc9c-8e694d46ed9e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:28.715564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b356c7-6c1b-4223-9ac2-d8f0fdfb3881 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:28.731353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:28.753887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2b19a-9b5d-4471-afa4-5fdb1a1d29ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:28.754560 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2b19a-9b5d-4471-afa4-5fdb1a1d29ca None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:28.760141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2b19a-9b5d-4471-afa4-5fdb1a1d29ca None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:28.760141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de2b19a-9b5d-4471-afa4-5fdb1a1d29ca None admin] PATH_INFO: `/v3/users/a97db02df8194b688f0ad261c5f615ec` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:28.769837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2b19a-9b5d-4471-afa4-5fdb1a1d29ca None admin] RBAC: Authorizing `identity:get_user(user_id=a97db02df8194b688f0ad261c5f615ec)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:28.771372 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de2b19a-9b5d-4471-afa4-5fdb1a1d29ca None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:28.773215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 96/190] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:28 2022] GET /identity/v3/users/a97db02df8194b688f0ad261c5f615ec => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:28.773983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2b19a-9b5d-4471-afa4-5fdb1a1d29ca None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:28.809397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c90bdb-46db-4ebc-bc9c-8e694d46ed9e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:28.846650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81706d69-1747-4b53-bfaa-b38bff325cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:28.847254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706d69-1747-4b53-bfaa-b38bff325cc4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:28.847525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706d69-1747-4b53-bfaa-b38bff325cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:28.847794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81706d69-1747-4b53-bfaa-b38bff325cc4 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:28.848448 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81706d69-1747-4b53-bfaa-b38bff325cc4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:28.848735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81706d69-1747-4b53-bfaa-b38bff325cc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:28.850122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81706d69-1747-4b53-bfaa-b38bff325cc4 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:28.860361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 95/191] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:33:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:28.861181 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81706d69-1747-4b53-bfaa-b38bff325cc4 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:28.899604 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de2b19a-9b5d-4471-afa4-5fdb1a1d29ca None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:28.923714 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:28.937626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f556b9-4eae-4cd6-9897-0fbbf6b429a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:28.937626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f556b9-4eae-4cd6-9897-0fbbf6b429a7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:28.937626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f556b9-4eae-4cd6-9897-0fbbf6b429a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:28.937626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f556b9-4eae-4cd6-9897-0fbbf6b429a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:28.938078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f556b9-4eae-4cd6-9897-0fbbf6b429a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:28.938644 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f556b9-4eae-4cd6-9897-0fbbf6b429a7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:28.940403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 97/192] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:28.942952 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f556b9-4eae-4cd6-9897-0fbbf6b429a7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:28.962591 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:28.970681 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81706d69-1747-4b53-bfaa-b38bff325cc4 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:29.007944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5f836-1e6c-434f-87d0-43dec32fc71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:29.008684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5f836-1e6c-434f-87d0-43dec32fc71c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:29.008958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5f836-1e6c-434f-87d0-43dec32fc71c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:29.009349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae5f836-1e6c-434f-87d0-43dec32fc71c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:29.048942 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f556b9-4eae-4cd6-9897-0fbbf6b429a7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:29.080856 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5f836-1e6c-434f-87d0-43dec32fc71c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:29.082529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae5f836-1e6c-434f-87d0-43dec32fc71c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:29.084400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c9660-576b-4ca3-b446-c2005935d11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:29.084972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c9660-576b-4ca3-b446-c2005935d11f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:29.085312 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c9660-576b-4ca3-b446-c2005935d11f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:29.085597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c9660-576b-4ca3-b446-c2005935d11f None admin] PATH_INFO: `/v3/users/a97db02df8194b688f0ad261c5f615ec` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:29.086791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c9660-576b-4ca3-b446-c2005935d11f None admin] RBAC: Authorizing `identity:get_user(user_id=a97db02df8194b688f0ad261c5f615ec)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:29.088219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c9660-576b-4ca3-b446-c2005935d11f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:29.089825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 98/193] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:28 2022] GET /identity/v3/users/a97db02df8194b688f0ad261c5f615ec => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:29.090576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c9660-576b-4ca3-b446-c2005935d11f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:29.155684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:29.180205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:29.187281 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 96/194] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:29.187972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5f836-1e6c-434f-87d0-43dec32fc71c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:29.198246 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c9660-576b-4ca3-b446-c2005935d11f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:29.232522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae8609-24d2-4514-8d34-8e078c12593d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:29.233238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae8609-24d2-4514-8d34-8e078c12593d None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:29.233584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae8609-24d2-4514-8d34-8e078c12593d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:29.233950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae8609-24d2-4514-8d34-8e078c12593d None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:29.234590 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae8609-24d2-4514-8d34-8e078c12593d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:29.236008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae8609-24d2-4514-8d34-8e078c12593d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:29.287593 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae5f836-1e6c-434f-87d0-43dec32fc71c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:29.319024 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 99/195] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:33:29 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:33:29.319024 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a242a8-e07c-4163-b76b-5a568943f580 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:29.319024 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a242a8-e07c-4163-b76b-5a568943f580 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:29.330632 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b478d50-279f-48ae-9427-351cb97ebac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:29.330721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a242a8-e07c-4163-b76b-5a568943f580 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:29.330789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b478d50-279f-48ae-9427-351cb97ebac4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:29.330789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b478d50-279f-48ae-9427-351cb97ebac4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:29.330789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b478d50-279f-48ae-9427-351cb97ebac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:29.330968 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 100/196] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:29.330968 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a242a8-e07c-4163-b76b-5a568943f580 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:29.395810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:29.405691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b478d50-279f-48ae-9427-351cb97ebac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:29.407201 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b478d50-279f-48ae-9427-351cb97ebac4 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:29.442372 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a242a8-e07c-4163-b76b-5a568943f580 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:29.457758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:29.479384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3c66e-b625-4285-9fca-28b770716a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:29.480370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3c66e-b625-4285-9fca-28b770716a20 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:29.480810 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3c66e-b625-4285-9fca-28b770716a20 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:29.481235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3c66e-b625-4285-9fca-28b770716a20 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:29.482850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3c66e-b625-4285-9fca-28b770716a20 None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:29.484457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3c66e-b625-4285-9fca-28b770716a20 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:29.486274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 101/197] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:29 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:29.487461 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22fefa-cf6a-4627-9eb2-cfab82a2ce9b None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:29.487744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22fefa-cf6a-4627-9eb2-cfab82a2ce9b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:29.488008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22fefa-cf6a-4627-9eb2-cfab82a2ce9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:29.508750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0e22fefa-cf6a-4627-9eb2-cfab82a2ce9b None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:29.509970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0e22fefa-cf6a-4627-9eb2-cfab82a2ce9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:29.529177 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 97/198] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:29.530314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b478d50-279f-48ae-9427-351cb97ebac4 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:29.588833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 102/199] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:29.590294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22fefa-cf6a-4627-9eb2-cfab82a2ce9b None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:29.642556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:29.652435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b478d50-279f-48ae-9427-351cb97ebac4 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:29.695883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0be22-45fa-4d72-a6c0-bce2c15894fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:29.696410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0be22-45fa-4d72-a6c0-bce2c15894fb None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:29.696682 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0be22-45fa-4d72-a6c0-bce2c15894fb None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:29.696949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c0be22-45fa-4d72-a6c0-bce2c15894fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:29.697776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0be22-45fa-4d72-a6c0-bce2c15894fb None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552, user.id=a97db02df8194b688f0ad261c5f615ec) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:29.698126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0be22-45fa-4d72-a6c0-bce2c15894fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:29.699689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c0be22-45fa-4d72-a6c0-bce2c15894fb None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:29.707212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 98/200] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:29 2022] GET /identity/v3/role_assignments?user.id=a97db02df8194b688f0ad261c5f615ec&scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:29.715771 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22fefa-cf6a-4627-9eb2-cfab82a2ce9b None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:29.731086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:29.755606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3b868-fa31-422e-86a8-b73572a3a3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:29.756256 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3b868-fa31-422e-86a8-b73572a3a3d5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:29.756529 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3b868-fa31-422e-86a8-b73572a3a3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:29.756807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3b868-fa31-422e-86a8-b73572a3a3d5 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:29.758336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3b868-fa31-422e-86a8-b73572a3a3d5 None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:29.760101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3b868-fa31-422e-86a8-b73572a3a3d5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:29.761771 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 103/201] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:29 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:29.768459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0be22-45fa-4d72-a6c0-bce2c15894fb None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:29.886147 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c0be22-45fa-4d72-a6c0-bce2c15894fb None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:29.905060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3b868-fa31-422e-86a8-b73572a3a3d5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:29.928672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ab8d9-b7ea-4d91-9f87-80cd48a33c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:29.929434 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ab8d9-b7ea-4d91-9f87-80cd48a33c86 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:29.929718 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ab8d9-b7ea-4d91-9f87-80cd48a33c86 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:29.930023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ab8d9-b7ea-4d91-9f87-80cd48a33c86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:29.931766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ab8d9-b7ea-4d91-9f87-80cd48a33c86 None admin] RBAC: Adding query filter params (role.id=36b107b4892f47f8af6b65e1d1d94bb0, scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552, user.id=a97db02df8194b688f0ad261c5f615ec) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:29.931766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ab8d9-b7ea-4d91-9f87-80cd48a33c86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:29.932583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0ab8d9-b7ea-4d91-9f87-80cd48a33c86 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:29.939571 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 99/202] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:29 2022] GET /identity/v3/role_assignments?user.id=a97db02df8194b688f0ad261c5f615ec&scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552&role.id=36b107b4892f47f8af6b65e1d1d94bb0 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:30.024971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3b868-fa31-422e-86a8-b73572a3a3d5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:30.070474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5f5e8-a296-4095-acbe-ce70fa245563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:30.071383 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5f5e8-a296-4095-acbe-ce70fa245563 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:30.071660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5f5e8-a296-4095-acbe-ce70fa245563 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:30.071930 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5f5e8-a296-4095-acbe-ce70fa245563 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:30.083436 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5f5e8-a296-4095-acbe-ce70fa245563 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-21b5f5e8-a296-4095-acbe-ce70fa245563 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:30.088496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:30.089456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:30.090287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:30.090287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:30.090287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:30.090287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:30.090287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:30.090287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:30.090287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:30.090287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:30.090287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:30.090287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:30.090287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:30.090287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:30.090287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 104/203] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:33:30.096551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ab8d9-b7ea-4d91-9f87-80cd48a33c86 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:30.135215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:30.228447 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ab8d9-b7ea-4d91-9f87-80cd48a33c86 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:30.276356 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d569b56-ca28-4243-b696-8438cf7308a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:30.276888 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d569b56-ca28-4243-b696-8438cf7308a1 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:30.277166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d569b56-ca28-4243-b696-8438cf7308a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:30.277439 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d569b56-ca28-4243-b696-8438cf7308a1 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:30.278137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d569b56-ca28-4243-b696-8438cf7308a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:30.278447 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d569b56-ca28-4243-b696-8438cf7308a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:30.280453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d569b56-ca28-4243-b696-8438cf7308a1 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:30.292988 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 100/204] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:30.299487 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5f5e8-a296-4095-acbe-ce70fa245563 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:30.342879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:30.419531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5f5e8-a296-4095-acbe-ce70fa245563 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:30.472406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cd0ef-5687-4cba-bfa9-bf1f94320b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:30.473050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cd0ef-5687-4cba-bfa9-bf1f94320b77 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:30.473325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cd0ef-5687-4cba-bfa9-bf1f94320b77 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:30.473592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558cd0ef-5687-4cba-bfa9-bf1f94320b77 None admin] PATH_INFO: `/v3/users/043f31f4faa940b7a300612a90db95ba` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:30.474752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cd0ef-5687-4cba-bfa9-bf1f94320b77 None admin] RBAC: Authorizing `identity:get_user(user_id=043f31f4faa940b7a300612a90db95ba)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:30.476190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558cd0ef-5687-4cba-bfa9-bf1f94320b77 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:30.478180 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 105/205] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:30 2022] GET /identity/v3/users/043f31f4faa940b7a300612a90db95ba => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:30.484159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d569b56-ca28-4243-b696-8438cf7308a1 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:30.606502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:30.624807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d569b56-ca28-4243-b696-8438cf7308a1 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:30.659817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adface24-f81f-4c28-815a-a234f4a02ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:30.660477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adface24-f81f-4c28-815a-a234f4a02ec8 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:30.660758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adface24-f81f-4c28-815a-a234f4a02ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:30.661032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adface24-f81f-4c28-815a-a234f4a02ec8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:30.744248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adface24-f81f-4c28-815a-a234f4a02ec8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:30.745785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adface24-f81f-4c28-815a-a234f4a02ec8 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:30.836438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:30.852433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 101/206] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:30.858923 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cd0ef-5687-4cba-bfa9-bf1f94320b77 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:30.965270 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558cd0ef-5687-4cba-bfa9-bf1f94320b77 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:30.975283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:31.000496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31159e4f-63c7-45fa-b7a0-4f2b65844096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:31.001127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31159e4f-63c7-45fa-b7a0-4f2b65844096 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:31.001405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31159e4f-63c7-45fa-b7a0-4f2b65844096 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:31.001672 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31159e4f-63c7-45fa-b7a0-4f2b65844096 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:31.002871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31159e4f-63c7-45fa-b7a0-4f2b65844096 None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:31.004400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31159e4f-63c7-45fa-b7a0-4f2b65844096 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:31.010213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 106/207] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:30 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:31.012214 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adface24-f81f-4c28-815a-a234f4a02ec8 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:31.131858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adface24-f81f-4c28-815a-a234f4a02ec8 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:31.166468 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea088a-1417-4d84-a37c-ab6428defd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:31.167072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea088a-1417-4d84-a37c-ab6428defd9f None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:31.167346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea088a-1417-4d84-a37c-ab6428defd9f None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:31.167614 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea088a-1417-4d84-a37c-ab6428defd9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:31.168331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea088a-1417-4d84-a37c-ab6428defd9f None admin] RBAC: Adding query filter params (role.id=35aa74738f2d4844af99831724f20d39, scope.project.id=07768f903a224486a253fc7acc9c3360, user.id=043f31f4faa940b7a300612a90db95ba) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:31.168634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea088a-1417-4d84-a37c-ab6428defd9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:31.170086 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea088a-1417-4d84-a37c-ab6428defd9f None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:31.180429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 102/208] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:31 2022] GET /identity/v3/role_assignments?user.id=043f31f4faa940b7a300612a90db95ba&scope.project.id=07768f903a224486a253fc7acc9c3360&role.id=35aa74738f2d4844af99831724f20d39 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:32.226636 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e78f43-ac78-4817-ac92-c2be6cec1917 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:32.227123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e78f43-ac78-4817-ac92-c2be6cec1917 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:32.227123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e78f43-ac78-4817-ac92-c2be6cec1917 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:32.228706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 107/209] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:32.234480 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c36e30-3691-483b-ab5a-422a10f4dbc8 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:32.234946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c36e30-3691-483b-ab5a-422a10f4dbc8 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:32.235217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c36e30-3691-483b-ab5a-422a10f4dbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:32.256458 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b4c36e30-3691-483b-ab5a-422a10f4dbc8 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:32.257662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b4c36e30-3691-483b-ab5a-422a10f4dbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:32.265586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ec231-fa6e-407f-b0fb-d9352130c3ce None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:32.265936 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ec231-fa6e-407f-b0fb-d9352130c3ce None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:32.266151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965ec231-fa6e-407f-b0fb-d9352130c3ce None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:32.267827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 108/210] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:32.273940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0b5e4-ad61-4e34-8207-dca1a22ba892 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:32.274253 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0b5e4-ad61-4e34-8207-dca1a22ba892 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:32.274492 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b0b5e4-ad61-4e34-8207-dca1a22ba892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:32.294461 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f6b0b5e4-ad61-4e34-8207-dca1a22ba892 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:32.296384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f6b0b5e4-ad61-4e34-8207-dca1a22ba892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:32.310202 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:32.331698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 103/211] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:32.372143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 109/212] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:32.392085 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03d937-39df-4fe1-a6f6-c3714e429085 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:32.392717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03d937-39df-4fe1-a6f6-c3714e429085 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:32.393194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce03d937-39df-4fe1-a6f6-c3714e429085 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:32.395247 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 104/213] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:32.401293 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8ebb-2d00-4114-878f-ddf269d51fc0 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:32.401981 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8ebb-2d00-4114-878f-ddf269d51fc0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:32.402480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8ebb-2d00-4114-878f-ddf269d51fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:32.422790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5e7d8ebb-2d00-4114-878f-ddf269d51fc0 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:32.424345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5e7d8ebb-2d00-4114-878f-ddf269d51fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:32.498180 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 110/214] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:32.515865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2aade8-f304-4be2-be6c-2ebabc66b8d7 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:32.516147 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2aade8-f304-4be2-be6c-2ebabc66b8d7 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:32.516420 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2aade8-f304-4be2-be6c-2ebabc66b8d7 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:32.518097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 105/215] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:32.523272 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62f97c-84fa-4225-ae31-93abeca23fa2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:32.523553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62f97c-84fa-4225-ae31-93abeca23fa2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:32.523823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62f97c-84fa-4225-ae31-93abeca23fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:32.542642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-fc62f97c-84fa-4225-ae31-93abeca23fa2 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:32.544000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fc62f97c-84fa-4225-ae31-93abeca23fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:32.573103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:32.611100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 111/216] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:32.649543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2aade8-f304-4be2-be6c-2ebabc66b8d7 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:32.652194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62f97c-84fa-4225-ae31-93abeca23fa2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:32.761997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2aade8-f304-4be2-be6c-2ebabc66b8d7 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:32.785656 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62f97c-84fa-4225-ae31-93abeca23fa2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:32.796759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f7581-421e-40b5-8961-ffa9ba4f029b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:32.797392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f7581-421e-40b5-8961-ffa9ba4f029b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:32.797681 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f7581-421e-40b5-8961-ffa9ba4f029b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:32.797979 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4f7581-421e-40b5-8961-ffa9ba4f029b None admin] PATH_INFO: `/v3/domains/default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:32.799056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f7581-421e-40b5-8961-ffa9ba4f029b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:32.800496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4f7581-421e-40b5-8961-ffa9ba4f029b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:32.806408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 106/217] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:32.807225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f7581-421e-40b5-8961-ffa9ba4f029b None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:32.821015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0199be3-3505-4662-951f-a327c551c63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:32.821645 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0199be3-3505-4662-951f-a327c551c63f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:32.821957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0199be3-3505-4662-951f-a327c551c63f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:32.822230 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0199be3-3505-4662-951f-a327c551c63f None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:32.832848 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0199be3-3505-4662-951f-a327c551c63f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-b0199be3-3505-4662-951f-a327c551c63f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:32.837597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:32.838502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:32.839290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:32.839290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:32.839290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:32.839290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:32.839290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:32.839290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:32.839290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:32.839290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:32.839290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:32.839290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:32.839290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:32.839290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:32.839290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 112/218] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:33:32.840022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0199be3-3505-4662-951f-a327c551c63f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:32.862820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:32.878788 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:32.914386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4f7581-421e-40b5-8961-ffa9ba4f029b None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:32.948099 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb4905-bf83-499e-87cd-692f01d9afe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:32.948714 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb4905-bf83-499e-87cd-692f01d9afe7 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:32.948985 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb4905-bf83-499e-87cd-692f01d9afe7 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:32.949250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb4905-bf83-499e-87cd-692f01d9afe7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:32.950322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0199be3-3505-4662-951f-a327c551c63f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:32.959230 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb4905-bf83-499e-87cd-692f01d9afe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-21bb4905-bf83-499e-87cd-692f01d9afe7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:32.963984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:32.965042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:32.965972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:32.965972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:32.965972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:32.965972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:32.965972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:32.965972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:32.965972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:32.965972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:32.965972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:32.965972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:32.965972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:32.965972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:32.965972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 107/219] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:33:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:32.965972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb4905-bf83-499e-87cd-692f01d9afe7 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:32.987134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ae666-7464-409e-9172-82356b5fa7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:32.987739 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ae666-7464-409e-9172-82356b5fa7c6 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:32.988123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ae666-7464-409e-9172-82356b5fa7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:32.988391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ae666-7464-409e-9172-82356b5fa7c6 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:32.989178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ae666-7464-409e-9172-82356b5fa7c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:32.990640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ae666-7464-409e-9172-82356b5fa7c6 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:33.061506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb4905-bf83-499e-87cd-692f01d9afe7 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:33.068624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:33.094476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee56106-5745-4bfd-b65a-a6e9ba53460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:33.095067 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee56106-5745-4bfd-b65a-a6e9ba53460b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:33.095342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee56106-5745-4bfd-b65a-a6e9ba53460b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:33.095609 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee56106-5745-4bfd-b65a-a6e9ba53460b None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:33.096286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee56106-5745-4bfd-b65a-a6e9ba53460b None admin] RBAC: Adding query filter params (name=member) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:33.096558 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee56106-5745-4bfd-b65a-a6e9ba53460b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:33.097424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 113/220] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:33:32 2022] POST /identity/v3/users => generated 297 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:33:33.097947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee56106-5745-4bfd-b65a-a6e9ba53460b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:33.101480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ae666-7464-409e-9172-82356b5fa7c6 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:33.106482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 108/221] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:33.106482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee56106-5745-4bfd-b65a-a6e9ba53460b None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:33.201944 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:33.221835 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ae666-7464-409e-9172-82356b5fa7c6 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:33.225945 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee56106-5745-4bfd-b65a-a6e9ba53460b None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:33.259844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5719e-792e-4d7a-bf39-b3f7ab0f638a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:33.260595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5719e-792e-4d7a-bf39-b3f7ab0f638a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:33.260762 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5719e-792e-4d7a-bf39-b3f7ab0f638a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:33.261074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5719e-792e-4d7a-bf39-b3f7ab0f638a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:33.262958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d716876-2371-48f4-9462-2356001d0891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:33.263572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d716876-2371-48f4-9462-2356001d0891 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:33.263850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d716876-2371-48f4-9462-2356001d0891 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:33.264278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d716876-2371-48f4-9462-2356001d0891 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:33.264952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d716876-2371-48f4-9462-2356001d0891 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:33.265253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d716876-2371-48f4-9462-2356001d0891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:33.266654 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d716876-2371-48f4-9462-2356001d0891 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:33.273541 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5719e-792e-4d7a-bf39-b3f7ab0f638a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:33.275297 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 109/222] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-9ac5719e-792e-4d7a-bf39-b3f7ab0f638a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:33.278803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:33.280202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:33.280992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:33.280992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:33.280992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:33.280992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:33.280992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:33.280992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:33.280992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:33.280992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:33.280992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:33.280992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:33.280992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:33.280992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:33.280992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 114/223] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:33:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:33.281650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d716876-2371-48f4-9462-2356001d0891 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:33.281718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5719e-792e-4d7a-bf39-b3f7ab0f638a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:33.320161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:33.400959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5719e-792e-4d7a-bf39-b3f7ab0f638a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:33.403038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d716876-2371-48f4-9462-2356001d0891 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:33.447447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e19230-3738-4792-8ee6-ca26dbceb3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:33.448064 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e19230-3738-4792-8ee6-ca26dbceb3aa None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:33.448340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e19230-3738-4792-8ee6-ca26dbceb3aa None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:33.448608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e19230-3738-4792-8ee6-ca26dbceb3aa None admin] PATH_INFO: `/v3/users/a97db02df8194b688f0ad261c5f615ec` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:33.450045 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e19230-3738-4792-8ee6-ca26dbceb3aa None admin] RBAC: Authorizing `identity:get_user(user_id=a97db02df8194b688f0ad261c5f615ec)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:33.451549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e19230-3738-4792-8ee6-ca26dbceb3aa None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:33.453142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 115/224] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:33 2022] GET /identity/v3/users/a97db02df8194b688f0ad261c5f615ec => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:33.454649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05abeb-b43c-484b-81bd-dd58560c5ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:33.455294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05abeb-b43c-484b-81bd-dd58560c5ac8 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:33.455570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05abeb-b43c-484b-81bd-dd58560c5ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:33.455838 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c05abeb-b43c-484b-81bd-dd58560c5ac8 None admin] PATH_INFO: `/v3/users/b9e128fc07ef420e94e299b50d370327` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:33.457196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e19230-3738-4792-8ee6-ca26dbceb3aa None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:33.473700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:33.477196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05abeb-b43c-484b-81bd-dd58560c5ac8 None admin] RBAC: Authorizing `identity:get_user(user_id=b9e128fc07ef420e94e299b50d370327)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:33.479062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c05abeb-b43c-484b-81bd-dd58560c5ac8 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:33.481344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 110/225] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:33 2022] GET /identity/v3/users/b9e128fc07ef420e94e299b50d370327 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:33.482064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05abeb-b43c-484b-81bd-dd58560c5ac8 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:33.564056 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e19230-3738-4792-8ee6-ca26dbceb3aa None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:33.599775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568ad5b-400b-42d1-b426-48ffd76cce4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:33.600298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568ad5b-400b-42d1-b426-48ffd76cce4f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:33.600580 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568ad5b-400b-42d1-b426-48ffd76cce4f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:33.600927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4568ad5b-400b-42d1-b426-48ffd76cce4f None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:33.601587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568ad5b-400b-42d1-b426-48ffd76cce4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:33.601955 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568ad5b-400b-42d1-b426-48ffd76cce4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:33.603384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4568ad5b-400b-42d1-b426-48ffd76cce4f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:33.604243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:33.612397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 116/226] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:33:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:33.613307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568ad5b-400b-42d1-b426-48ffd76cce4f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:33.620208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c05abeb-b43c-484b-81bd-dd58560c5ac8 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:33.669226 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287444e-7e9b-49e6-b252-c33103a2a0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:33.670220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287444e-7e9b-49e6-b252-c33103a2a0f1 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:33.671201 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287444e-7e9b-49e6-b252-c33103a2a0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:33.672161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1287444e-7e9b-49e6-b252-c33103a2a0f1 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:33.673686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287444e-7e9b-49e6-b252-c33103a2a0f1 None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:33.676260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1287444e-7e9b-49e6-b252-c33103a2a0f1 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:33.683656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 111/227] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:33 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:33.683656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287444e-7e9b-49e6-b252-c33103a2a0f1 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:33.698775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:33.730063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4568ad5b-400b-42d1-b426-48ffd76cce4f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:33.771459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b96cc3-1042-469f-9a49-11c0159609d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:33.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b96cc3-1042-469f-9a49-11c0159609d5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:33.772779 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b96cc3-1042-469f-9a49-11c0159609d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:33.773325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b96cc3-1042-469f-9a49-11c0159609d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:33.814223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1287444e-7e9b-49e6-b252-c33103a2a0f1 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:33.851020 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84489082-1964-446d-b193-2d260fdb6424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:33.851717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84489082-1964-446d-b193-2d260fdb6424 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:33.852057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84489082-1964-446d-b193-2d260fdb6424 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:33.852395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84489082-1964-446d-b193-2d260fdb6424 None admin] PATH_INFO: `/v3/users/a97db02df8194b688f0ad261c5f615ec` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:33.853767 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84489082-1964-446d-b193-2d260fdb6424 None admin] RBAC: Authorizing `identity:get_user(user_id=a97db02df8194b688f0ad261c5f615ec)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:33.855307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84489082-1964-446d-b193-2d260fdb6424 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:33.857414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 112/228] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:33 2022] GET /identity/v3/users/a97db02df8194b688f0ad261c5f615ec => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:33.858860 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84489082-1964-446d-b193-2d260fdb6424 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:33.864993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b96cc3-1042-469f-9a49-11c0159609d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:33.866516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b96cc3-1042-469f-9a49-11c0159609d5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:33.890161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:33.957911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:33.971181 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84489082-1964-446d-b193-2d260fdb6424 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:33.997152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 117/229] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:33.998182 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b96cc3-1042-469f-9a49-11c0159609d5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:34.008984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c7b89-6df0-419b-bbbd-cf492229cdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:34.010034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c7b89-6df0-419b-bbbd-cf492229cdd6 None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:34.010507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c7b89-6df0-419b-bbbd-cf492229cdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:34.011064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11c7b89-6df0-419b-bbbd-cf492229cdd6 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552/users/a97db02df8194b688f0ad261c5f615ec/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:34.013315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c7b89-6df0-419b-bbbd-cf492229cdd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5fbdcb3c4884d59ab798fc1c9752552, user_id=a97db02df8194b688f0ad261c5f615ec, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:34.015128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11c7b89-6df0-419b-bbbd-cf492229cdd6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:34.103221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b96cc3-1042-469f-9a49-11c0159609d5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:34.144135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 113/230] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:33:33 2022] PUT /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552/users/a97db02df8194b688f0ad261c5f615ec/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:34.144750 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c7b89-6df0-419b-bbbd-cf492229cdd6 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:34.147668 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420eec07-93fd-443e-af14-1e6bd79abbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:34.150734 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420eec07-93fd-443e-af14-1e6bd79abbe0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:34.150734 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420eec07-93fd-443e-af14-1e6bd79abbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:34.150734 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420eec07-93fd-443e-af14-1e6bd79abbe0 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:34.150734 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420eec07-93fd-443e-af14-1e6bd79abbe0 None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:34.151546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420eec07-93fd-443e-af14-1e6bd79abbe0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:34.153616 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 118/231] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:33 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:34.162967 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420eec07-93fd-443e-af14-1e6bd79abbe0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:34.263512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:34.284668 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:34.287254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11c7b89-6df0-419b-bbbd-cf492229cdd6 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:34.329635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba383d5-59c4-4782-ae55-65070d3351bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:34.330299 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba383d5-59c4-4782-ae55-65070d3351bd None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:34.330577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba383d5-59c4-4782-ae55-65070d3351bd None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:34.330850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba383d5-59c4-4782-ae55-65070d3351bd None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:34.332112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba383d5-59c4-4782-ae55-65070d3351bd None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:34.333588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba383d5-59c4-4782-ae55-65070d3351bd None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:34.335426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 114/232] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:34 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:34.341183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba383d5-59c4-4782-ae55-65070d3351bd None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:34.384906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420eec07-93fd-443e-af14-1e6bd79abbe0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:34.425446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b287431-ed57-41bc-942b-59959acb1cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:34.426213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287431-ed57-41bc-942b-59959acb1cb5 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:34.426488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287431-ed57-41bc-942b-59959acb1cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:34.426758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287431-ed57-41bc-942b-59959acb1cb5 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552/users/a97db02df8194b688f0ad261c5f615ec/roles/36b107b4892f47f8af6b65e1d1d94bb0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:34.430404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b287431-ed57-41bc-942b-59959acb1cb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5fbdcb3c4884d59ab798fc1c9752552, user_id=a97db02df8194b688f0ad261c5f615ec, role_id=36b107b4892f47f8af6b65e1d1d94bb0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:34.431951 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b287431-ed57-41bc-942b-59959acb1cb5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:34.457980 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba383d5-59c4-4782-ae55-65070d3351bd None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:34.493790 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dd50f2-9d4d-4d91-b5d5-47857d437f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:34.499880 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd50f2-9d4d-4d91-b5d5-47857d437f23 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:34.499880 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd50f2-9d4d-4d91-b5d5-47857d437f23 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:34.499880 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dd50f2-9d4d-4d91-b5d5-47857d437f23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:34.499880 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd50f2-9d4d-4d91-b5d5-47857d437f23 None admin] RBAC: Adding query filter params (role.id=efa4ca02b2ff4ad6b14d501d4d79bbea, scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552, user.id=b9e128fc07ef420e94e299b50d370327) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:34.499880 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd50f2-9d4d-4d91-b5d5-47857d437f23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:34.499880 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dd50f2-9d4d-4d91-b5d5-47857d437f23 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:34.504582 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 115/233] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:34 2022] GET /identity/v3/role_assignments?user.id=b9e128fc07ef420e94e299b50d370327&scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552&role.id=efa4ca02b2ff4ad6b14d501d4d79bbea => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:34.569321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 119/234] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:33:34 2022] PUT /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552/users/a97db02df8194b688f0ad261c5f615ec/roles/36b107b4892f47f8af6b65e1d1d94bb0 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:35.697300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03af22-eef0-49ab-a0b7-4c3f2bd22deb None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:35.698375 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03af22-eef0-49ab-a0b7-4c3f2bd22deb None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:35.698850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03af22-eef0-49ab-a0b7-4c3f2bd22deb None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:35.700394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 116/235] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:35.705928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de811ba-dd4d-4c6d-9b78-8fdb0ef776e4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:35.706752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de811ba-dd4d-4c6d-9b78-8fdb0ef776e4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:35.707218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de811ba-dd4d-4c6d-9b78-8fdb0ef776e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:35.730174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb7876-d213-44bd-b663-ba87c17abde1 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:35.730775 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb7876-d213-44bd-b663-ba87c17abde1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:35.731236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb7876-d213-44bd-b663-ba87c17abde1 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:35.732843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 117/236] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:33:35.735512 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1de811ba-dd4d-4c6d-9b78-8fdb0ef776e4 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:35.736924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1de811ba-dd4d-4c6d-9b78-8fdb0ef776e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:35.738213 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3768255-378f-4b0f-82aa-7583fb72ce7f None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:35.738495 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3768255-378f-4b0f-82aa-7583fb72ce7f None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:35.738770 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3768255-378f-4b0f-82aa-7583fb72ce7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:35.763111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-c3768255-378f-4b0f-82aa-7583fb72ce7f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:35.765316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c3768255-378f-4b0f-82aa-7583fb72ce7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:35.786541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:35.851865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 118/237] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:35.863951 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 120/238] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:36.133734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3768255-378f-4b0f-82aa-7583fb72ce7f None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:36.244415 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3768255-378f-4b0f-82aa-7583fb72ce7f None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:36.278855 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa642e4-e979-4789-8429-bab52fd720f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:36.280360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa642e4-e979-4789-8429-bab52fd720f2 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:36.280839 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa642e4-e979-4789-8429-bab52fd720f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:36.281340 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa642e4-e979-4789-8429-bab52fd720f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:36.282658 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa642e4-e979-4789-8429-bab52fd720f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:36.284271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa642e4-e979-4789-8429-bab52fd720f2 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:36.286353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 119/239] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:36.291999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de811ba-dd4d-4c6d-9b78-8fdb0ef776e4 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:36.323295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:36.323470 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa642e4-e979-4789-8429-bab52fd720f2 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:36.358164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:36.396009 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de811ba-dd4d-4c6d-9b78-8fdb0ef776e4 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:36.430899 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d713c43-e0fe-4456-a21a-9b60907bcd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:36.431563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d713c43-e0fe-4456-a21a-9b60907bcd4f None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:36.431847 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d713c43-e0fe-4456-a21a-9b60907bcd4f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:36.432115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d713c43-e0fe-4456-a21a-9b60907bcd4f None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:36.432766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d713c43-e0fe-4456-a21a-9b60907bcd4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:36.434373 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d713c43-e0fe-4456-a21a-9b60907bcd4f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:36.498965 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa642e4-e979-4789-8429-bab52fd720f2 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:36.512694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 121/240] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:33:36 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:33:36.556336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69602025-7a34-4ec3-abaf-938752a4faaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:36.557046 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69602025-7a34-4ec3-abaf-938752a4faaa None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:36.557321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69602025-7a34-4ec3-abaf-938752a4faaa None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:36.557750 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69602025-7a34-4ec3-abaf-938752a4faaa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:36.575780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69602025-7a34-4ec3-abaf-938752a4faaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-69602025-7a34-4ec3-abaf-938752a4faaa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:36.581017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:36.581938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:36.582745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:36.582745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:36.582745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:36.582745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:36.582745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:36.582745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:36.582745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:36.582745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:36.582745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:36.582745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:36.582745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:36.582745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:36.582745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 120/241] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:36.589926 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d713c43-e0fe-4456-a21a-9b60907bcd4f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:36.703635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:36.724757 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d713c43-e0fe-4456-a21a-9b60907bcd4f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:36.773498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd84e8-a087-4260-a094-6698b376e65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:36.774184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd84e8-a087-4260-a094-6698b376e65e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:36.774465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd84e8-a087-4260-a094-6698b376e65e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:36.774744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd84e8-a087-4260-a094-6698b376e65e None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:36.775417 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd84e8-a087-4260-a094-6698b376e65e None admin] RBAC: Adding query filter params (name=reader) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:36.775725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd84e8-a087-4260-a094-6698b376e65e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:36.777325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd84e8-a087-4260-a094-6698b376e65e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:36.789399 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 122/242] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:36.796010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69602025-7a34-4ec3-abaf-938752a4faaa None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:36.901713 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:36.929610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69602025-7a34-4ec3-abaf-938752a4faaa None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:36.930342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55397abf-ada9-41f3-993e-61f1e315d461 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:36.930634 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55397abf-ada9-41f3-993e-61f1e315d461 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:36.930914 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55397abf-ada9-41f3-993e-61f1e315d461 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:36.932280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 123/243] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:33:36.938735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c94397-5f54-4119-a6d8-dd8448d78e4c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:36.939013 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c94397-5f54-4119-a6d8-dd8448d78e4c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:36.939287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c94397-5f54-4119-a6d8-dd8448d78e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:36.961640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a5c94397-5f54-4119-a6d8-dd8448d78e4c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:36.962835 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a5c94397-5f54-4119-a6d8-dd8448d78e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:36.970398 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be6161-a245-4410-89c0-881ac68cbe3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:36.971335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be6161-a245-4410-89c0-881ac68cbe3f None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:36.971836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be6161-a245-4410-89c0-881ac68cbe3f None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:36.972380 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be6161-a245-4410-89c0-881ac68cbe3f None admin] PATH_INFO: `/v3/users/63f30ef3af5e4c4dbdadc233888e3ccc` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:36.990423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be6161-a245-4410-89c0-881ac68cbe3f None admin] RBAC: Authorizing `identity:get_user(user_id=63f30ef3af5e4c4dbdadc233888e3ccc)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:36.992396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8be6161-a245-4410-89c0-881ac68cbe3f None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:36.994843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 121/244] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:36 2022] GET /identity/v3/users/63f30ef3af5e4c4dbdadc233888e3ccc => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:37.002056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be6161-a245-4410-89c0-881ac68cbe3f None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:37.056484 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 124/245] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:37.115954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8be6161-a245-4410-89c0-881ac68cbe3f None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:37.140891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473c774-8297-460a-92c4-9c9316eae19c None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:37.141178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473c774-8297-460a-92c4-9c9316eae19c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:37.141443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b473c774-8297-460a-92c4-9c9316eae19c None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:37.143083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 125/246] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:33:37.146948 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:37.148481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465b107-bad5-4896-b419-62db42993572 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:37.148860 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465b107-bad5-4896-b419-62db42993572 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:37.149139 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b465b107-bad5-4896-b419-62db42993572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:37.158962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7835b9-02af-4f4e-8f98-677ef70f5445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:37.159586 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7835b9-02af-4f4e-8f98-677ef70f5445 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:37.159856 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7835b9-02af-4f4e-8f98-677ef70f5445 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:37.160120 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7835b9-02af-4f4e-8f98-677ef70f5445 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:37.169385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b465b107-bad5-4896-b419-62db42993572 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:37.170666 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b465b107-bad5-4896-b419-62db42993572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:37.244525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7835b9-02af-4f4e-8f98-677ef70f5445 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:37.245993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7835b9-02af-4f4e-8f98-677ef70f5445 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:37.260434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 126/247] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:37.351883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 122/248] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:37.357530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465b107-bad5-4896-b419-62db42993572 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:37.365770 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7835b9-02af-4f4e-8f98-677ef70f5445 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:37.374197 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:37.405065 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:37.494473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7835b9-02af-4f4e-8f98-677ef70f5445 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:37.495706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b465b107-bad5-4896-b419-62db42993572 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:37.536849 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7600977-1552-4f19-8fde-f70a0260c28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:37.537492 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7600977-1552-4f19-8fde-f70a0260c28f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:37.537769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7600977-1552-4f19-8fde-f70a0260c28f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:37.538079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7600977-1552-4f19-8fde-f70a0260c28f None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:37.539977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7600977-1552-4f19-8fde-f70a0260c28f None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:37.540192 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7634552-c7b4-4963-ae31-a1744d9e4c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:37.540835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7634552-c7b4-4963-ae31-a1744d9e4c65 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:37.541110 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7634552-c7b4-4963-ae31-a1744d9e4c65 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:37.541379 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7634552-c7b4-4963-ae31-a1744d9e4c65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:37.541488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7600977-1552-4f19-8fde-f70a0260c28f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:37.543946 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 127/249] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:37 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:37.553180 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a971f60-17a8-4aa5-b89a-b99d8c8727a8 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:37.553592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a971f60-17a8-4aa5-b89a-b99d8c8727a8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:37.553770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a971f60-17a8-4aa5-b89a-b99d8c8727a8 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:37.559045 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 128/250] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:37 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:37.559846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a971f60-17a8-4aa5-b89a-b99d8c8727a8 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:37.569384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7634552-c7b4-4963-ae31-a1744d9e4c65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-e7634552-c7b4-4963-ae31-a1744d9e4c65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:37.577648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:37.578711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:37.579484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:37.579484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:37.579484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:37.579484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:37.579484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:37.579484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:37.579484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:37.579484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:37.579484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:37.579484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:37.579484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:37.579484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:37.579484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 123/251] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:33:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:33:37.580569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7721c-2139-4654-9b3b-38bce3b1bc21 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:37.580569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7721c-2139-4654-9b3b-38bce3b1bc21 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:37.580819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7721c-2139-4654-9b3b-38bce3b1bc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:37.603182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-10a7721c-2139-4654-9b3b-38bce3b1bc21 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:37.604413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-10a7721c-2139-4654-9b3b-38bce3b1bc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:37.668467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:37.680306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 124/252] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:37.681180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7721c-2139-4654-9b3b-38bce3b1bc21 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:37.700628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a971f60-17a8-4aa5-b89a-b99d8c8727a8 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:37.708311 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:37.746524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d6b39-3058-4ad6-8fcc-0ccbd2de8c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:37.747784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d6b39-3058-4ad6-8fcc-0ccbd2de8c3e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:37.748083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d6b39-3058-4ad6-8fcc-0ccbd2de8c3e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:37.748952 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d6b39-3058-4ad6-8fcc-0ccbd2de8c3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:37.749696 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d6b39-3058-4ad6-8fcc-0ccbd2de8c3e None admin] RBAC: Adding query filter params (role.id=35aa74738f2d4844af99831724f20d39, scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552, user.id=63f30ef3af5e4c4dbdadc233888e3ccc) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:37.750221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d6b39-3058-4ad6-8fcc-0ccbd2de8c3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:37.752757 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d6b39-3058-4ad6-8fcc-0ccbd2de8c3e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:37.759392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 129/253] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:37 2022] GET /identity/v3/role_assignments?user.id=63f30ef3af5e4c4dbdadc233888e3ccc&scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552&role.id=35aa74738f2d4844af99831724f20d39 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:37.760190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d6b39-3058-4ad6-8fcc-0ccbd2de8c3e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:37.843798 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7721c-2139-4654-9b3b-38bce3b1bc21 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:37.917215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80134eb-7ee4-43b6-b423-4fb575684fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:37.918291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80134eb-7ee4-43b6-b423-4fb575684fc7 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:37.919601 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80134eb-7ee4-43b6-b423-4fb575684fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:37.920058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80134eb-7ee4-43b6-b423-4fb575684fc7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:37.948177 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80134eb-7ee4-43b6-b423-4fb575684fc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-d80134eb-7ee4-43b6-b423-4fb575684fc7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:37.958443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:37.959357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:37.960180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:37.960180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:37.960180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:37.960180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:37.960180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:37.960180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:37.960180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:37.960180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:37.960180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:37.960180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:37.960180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:37.960180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:37.962654 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 125/254] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:37.969371 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80134eb-7ee4-43b6-b423-4fb575684fc7 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:37.971802 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d6b39-3058-4ad6-8fcc-0ccbd2de8c3e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:38.010675 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fc442-b390-483a-b291-a19fbd01bc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:38.011555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fc442-b390-483a-b291-a19fbd01bc4e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:38.011841 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fc442-b390-483a-b291-a19fbd01bc4e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:38.012212 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fc442-b390-483a-b291-a19fbd01bc4e None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:38.012985 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fc442-b390-483a-b291-a19fbd01bc4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:38.013376 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fc442-b390-483a-b291-a19fbd01bc4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:38.015562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fc442-b390-483a-b291-a19fbd01bc4e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:38.039159 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 130/255] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:38.047941 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fc442-b390-483a-b291-a19fbd01bc4e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:38.051636 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:38.143035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:38.162062 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fc442-b390-483a-b291-a19fbd01bc4e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:38.175792 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80134eb-7ee4-43b6-b423-4fb575684fc7 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:38.203507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a8233-33e9-4db8-9c3a-f31e6b0e6318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:38.204736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a8233-33e9-4db8-9c3a-f31e6b0e6318 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:38.206209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a8233-33e9-4db8-9c3a-f31e6b0e6318 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:38.206856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a8233-33e9-4db8-9c3a-f31e6b0e6318 None admin] PATH_INFO: `/v3/users/a97db02df8194b688f0ad261c5f615ec` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:38.209394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a8233-33e9-4db8-9c3a-f31e6b0e6318 None admin] RBAC: Authorizing `identity:get_user(user_id=a97db02df8194b688f0ad261c5f615ec)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:38.211262 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a8233-33e9-4db8-9c3a-f31e6b0e6318 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:38.214044 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 131/256] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:38 2022] GET /identity/v3/users/a97db02df8194b688f0ad261c5f615ec => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:38.221822 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a8233-33e9-4db8-9c3a-f31e6b0e6318 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:38.250251 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e5ff9-dc8e-4655-bb4a-2a4ccd477472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:38.251443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e5ff9-dc8e-4655-bb4a-2a4ccd477472 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:38.252791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e5ff9-dc8e-4655-bb4a-2a4ccd477472 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:38.254653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098e5ff9-dc8e-4655-bb4a-2a4ccd477472 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:38.255593 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e5ff9-dc8e-4655-bb4a-2a4ccd477472 None admin] RBAC: Adding query filter params (name=member) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:38.256097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e5ff9-dc8e-4655-bb4a-2a4ccd477472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:38.258224 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098e5ff9-dc8e-4655-bb4a-2a4ccd477472 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:38.281199 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 126/257] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:38.282041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e5ff9-dc8e-4655-bb4a-2a4ccd477472 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:38.362717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:38.421927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a8233-33e9-4db8-9c3a-f31e6b0e6318 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:38.459344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098e5ff9-dc8e-4655-bb4a-2a4ccd477472 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:38.501149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4721bb9-9d7c-4cf2-9b4d-dda9e1257cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:38.502143 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4721bb9-9d7c-4cf2-9b4d-dda9e1257cad None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:38.502652 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4721bb9-9d7c-4cf2-9b4d-dda9e1257cad None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:38.503124 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4721bb9-9d7c-4cf2-9b4d-dda9e1257cad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:38.515120 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a349be8-dc9c-4da4-b330-d97f88d8e03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:38.515763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a349be8-dc9c-4da4-b330-d97f88d8e03a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:38.516043 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a349be8-dc9c-4da4-b330-d97f88d8e03a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:38.516309 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a349be8-dc9c-4da4-b330-d97f88d8e03a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:38.520892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:38.539570 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a349be8-dc9c-4da4-b330-d97f88d8e03a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-2a349be8-dc9c-4da4-b330-d97f88d8e03a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:38.549276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:38.551208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:38.551994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:38.551994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:38.551994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:38.551994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:38.551994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:38.551994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:38.551994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:38.551994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:38.551994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:38.551994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:38.551994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:38.551994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:38.555386 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 132/258] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:33:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:38.557700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a349be8-dc9c-4da4-b330-d97f88d8e03a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:38.645829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4721bb9-9d7c-4cf2-9b4d-dda9e1257cad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:38.647647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4721bb9-9d7c-4cf2-9b4d-dda9e1257cad None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:38.685290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a349be8-dc9c-4da4-b330-d97f88d8e03a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:38.710054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:38.720941 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e68a44-d40d-4ed0-8c52-fbf6c0267378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:38.721835 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e68a44-d40d-4ed0-8c52-fbf6c0267378 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:38.722277 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e68a44-d40d-4ed0-8c52-fbf6c0267378 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:38.722705 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e68a44-d40d-4ed0-8c52-fbf6c0267378 None admin] PATH_INFO: `/v3/users/b9e128fc07ef420e94e299b50d370327` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:38.725413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e68a44-d40d-4ed0-8c52-fbf6c0267378 None admin] RBAC: Authorizing `identity:get_user(user_id=b9e128fc07ef420e94e299b50d370327)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:38.727721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e68a44-d40d-4ed0-8c52-fbf6c0267378 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:38.732440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 133/259] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:38 2022] GET /identity/v3/users/b9e128fc07ef420e94e299b50d370327 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:38.734299 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e68a44-d40d-4ed0-8c52-fbf6c0267378 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:38.823198 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 127/260] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:38.824043 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4721bb9-9d7c-4cf2-9b4d-dda9e1257cad None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:38.910167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:38.936388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e68a44-d40d-4ed0-8c52-fbf6c0267378 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:39.006765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949a519-060d-4938-8bfc-19977f7f1865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:39.007380 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949a519-060d-4938-8bfc-19977f7f1865 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:39.007656 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949a519-060d-4938-8bfc-19977f7f1865 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:39.007927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949a519-060d-4938-8bfc-19977f7f1865 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:39.008585 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949a519-060d-4938-8bfc-19977f7f1865 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:39.008884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949a519-060d-4938-8bfc-19977f7f1865 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:39.010301 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949a519-060d-4938-8bfc-19977f7f1865 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:39.015672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4721bb9-9d7c-4cf2-9b4d-dda9e1257cad None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:39.030882 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 134/261] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:33:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:39.031537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949a519-060d-4938-8bfc-19977f7f1865 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:39.083517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad6e84-4685-4c80-945b-fd4f1cfb2a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:39.084073 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad6e84-4685-4c80-945b-fd4f1cfb2a07 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:39.084348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad6e84-4685-4c80-945b-fd4f1cfb2a07 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:39.084618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ad6e84-4685-4c80-945b-fd4f1cfb2a07 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:39.086572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad6e84-4685-4c80-945b-fd4f1cfb2a07 None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:39.087710 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ad6e84-4685-4c80-945b-fd4f1cfb2a07 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:39.089904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 128/262] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:38 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:39.093846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad6e84-4685-4c80-945b-fd4f1cfb2a07 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:39.135797 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:39.208627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ad6e84-4685-4c80-945b-fd4f1cfb2a07 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:39.219429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949a519-060d-4938-8bfc-19977f7f1865 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:39.231531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:39.246376 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222525e4-d120-4388-8a3b-9b41b45f7d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:39.247323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222525e4-d120-4388-8a3b-9b41b45f7d42 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:39.247763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222525e4-d120-4388-8a3b-9b41b45f7d42 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:39.248770 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222525e4-d120-4388-8a3b-9b41b45f7d42 None admin] PATH_INFO: `/v3/users/a97db02df8194b688f0ad261c5f615ec` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:39.250189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222525e4-d120-4388-8a3b-9b41b45f7d42 None admin] RBAC: Authorizing `identity:get_user(user_id=a97db02df8194b688f0ad261c5f615ec)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:39.251859 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222525e4-d120-4388-8a3b-9b41b45f7d42 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:39.253775 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 129/263] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:39 2022] GET /identity/v3/users/a97db02df8194b688f0ad261c5f615ec => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:39.254831 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222525e4-d120-4388-8a3b-9b41b45f7d42 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:39.267542 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be9f63-b232-4598-8a3e-c03e9faee7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:39.268434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be9f63-b232-4598-8a3e-c03e9faee7ae None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:39.268725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be9f63-b232-4598-8a3e-c03e9faee7ae None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:39.268992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be9f63-b232-4598-8a3e-c03e9faee7ae None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:39.270143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be9f63-b232-4598-8a3e-c03e9faee7ae None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:39.271834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be9f63-b232-4598-8a3e-c03e9faee7ae None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:39.273181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 135/264] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:39 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:39.273949 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be9f63-b232-4598-8a3e-c03e9faee7ae None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:39.404449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be9f63-b232-4598-8a3e-c03e9faee7ae None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:39.431982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222525e4-d120-4388-8a3b-9b41b45f7d42 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:39.438291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:39.446142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc5bf6-55bf-44a1-acb6-dfd475088bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:39.446800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc5bf6-55bf-44a1-acb6-dfd475088bde None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:39.447078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc5bf6-55bf-44a1-acb6-dfd475088bde None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:39.447353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc5bf6-55bf-44a1-acb6-dfd475088bde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:39.491583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3127d2-8299-401d-a9dd-addd1ed67582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:39.492572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3127d2-8299-401d-a9dd-addd1ed67582 None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:39.492997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3127d2-8299-401d-a9dd-addd1ed67582 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:39.493461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3127d2-8299-401d-a9dd-addd1ed67582 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552/users/b9e128fc07ef420e94e299b50d370327/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:39.496562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3127d2-8299-401d-a9dd-addd1ed67582 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5fbdcb3c4884d59ab798fc1c9752552, user_id=b9e128fc07ef420e94e299b50d370327, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:39.499068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3127d2-8299-401d-a9dd-addd1ed67582 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:39.547860 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc5bf6-55bf-44a1-acb6-dfd475088bde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:39.549640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc5bf6-55bf-44a1-acb6-dfd475088bde None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:39.612348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 130/265] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:33:39 2022] PUT /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552/users/b9e128fc07ef420e94e299b50d370327/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:39.613491 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3127d2-8299-401d-a9dd-addd1ed67582 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:39.693365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:39.711531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 136/266] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:39.714713 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07681af1-ac84-42d0-957c-c1a01a68f488 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:39.716305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07681af1-ac84-42d0-957c-c1a01a68f488 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:39.719584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07681af1-ac84-42d0-957c-c1a01a68f488 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:39.721291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 137/267] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:39.722980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07681af1-ac84-42d0-957c-c1a01a68f488 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:39.777768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3127d2-8299-401d-a9dd-addd1ed67582 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:39.786708 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:39.829118 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee1aa9-d9e7-4326-8784-e5966044a428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:39.830421 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee1aa9-d9e7-4326-8784-e5966044a428 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:39.830979 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee1aa9-d9e7-4326-8784-e5966044a428 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:39.831451 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee1aa9-d9e7-4326-8784-e5966044a428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:39.833299 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee1aa9-d9e7-4326-8784-e5966044a428 None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552, user.id=a97db02df8194b688f0ad261c5f615ec) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:39.833951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee1aa9-d9e7-4326-8784-e5966044a428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:39.835916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee1aa9-d9e7-4326-8784-e5966044a428 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:39.843736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 131/268] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:39 2022] GET /identity/v3/role_assignments?user.id=a97db02df8194b688f0ad261c5f615ec&scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:39.850819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e55faa-97fd-4005-8f15-a337b4174206 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:39.851541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e55faa-97fd-4005-8f15-a337b4174206 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:39.852323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e55faa-97fd-4005-8f15-a337b4174206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:39.892349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-69e55faa-97fd-4005-8f15-a337b4174206 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:39.893572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-69e55faa-97fd-4005-8f15-a337b4174206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:39.980807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07681af1-ac84-42d0-957c-c1a01a68f488 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:40.015475 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 132/269] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:40.037092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5596d08-8662-4db1-85bd-e36e2da80f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:40.037744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5596d08-8662-4db1-85bd-e36e2da80f9c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:40.038051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5596d08-8662-4db1-85bd-e36e2da80f9c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:40.038322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5596d08-8662-4db1-85bd-e36e2da80f9c None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:40.039486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5596d08-8662-4db1-85bd-e36e2da80f9c None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:40.041125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5596d08-8662-4db1-85bd-e36e2da80f9c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:40.042945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 138/270] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:39 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:40.049630 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e55faa-97fd-4005-8f15-a337b4174206 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:40.095831 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:40.167519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e55faa-97fd-4005-8f15-a337b4174206 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:40.205404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada5374-dca6-43d4-babb-bf8e651b1c46 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:40.205703 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada5374-dca6-43d4-babb-bf8e651b1c46 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:40.206024 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fada5374-dca6-43d4-babb-bf8e651b1c46 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:40.207587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 139/271] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:33:40.207860 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0c56e-d143-42fe-814c-4beae5885a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:40.208528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0c56e-d143-42fe-814c-4beae5885a28 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:40.208873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0c56e-d143-42fe-814c-4beae5885a28 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:40.209150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0c56e-d143-42fe-814c-4beae5885a28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:40.209901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0c56e-d143-42fe-814c-4beae5885a28 None admin] RBAC: Adding query filter params (role.id=36b107b4892f47f8af6b65e1d1d94bb0, scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552, user.id=a97db02df8194b688f0ad261c5f615ec) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:40.210212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0c56e-d143-42fe-814c-4beae5885a28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:40.211784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f0c56e-d143-42fe-814c-4beae5885a28 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:40.212724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308e7e1-fcaf-4bfc-8075-983248eb217c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:40.213003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308e7e1-fcaf-4bfc-8075-983248eb217c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:40.213273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308e7e1-fcaf-4bfc-8075-983248eb217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:40.221429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 133/272] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:40 2022] GET /identity/v3/role_assignments?user.id=a97db02df8194b688f0ad261c5f615ec&scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552&role.id=36b107b4892f47f8af6b65e1d1d94bb0 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:40.250867 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1308e7e1-fcaf-4bfc-8075-983248eb217c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:40.252104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1308e7e1-fcaf-4bfc-8075-983248eb217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:40.351126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 140/273] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:40.397771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0c56e-d143-42fe-814c-4beae5885a28 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:40.525457 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:40.529168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0c56e-d143-42fe-814c-4beae5885a28 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:40.565990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c9715-07bf-4b43-af79-40bb883162fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:40.566626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c9715-07bf-4b43-af79-40bb883162fa None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:40.566905 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c9715-07bf-4b43-af79-40bb883162fa None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:40.567429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c9715-07bf-4b43-af79-40bb883162fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:40.573573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc20c3-72df-4cc0-b895-56de15bdd1d9 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:40.573791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc20c3-72df-4cc0-b895-56de15bdd1d9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:40.574094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc20c3-72df-4cc0-b895-56de15bdd1d9 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:40.575667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 141/274] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:33:40.578853 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c9715-07bf-4b43-af79-40bb883162fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:40.581161 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e278905-282c-4961-a21d-6de6870a8ad4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:40.581443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e278905-282c-4961-a21d-6de6870a8ad4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:40.581715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e278905-282c-4961-a21d-6de6870a8ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-f33c9715-07bf-4b43-af79-40bb883162fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:40.584452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:40.585326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:40.586539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:40.586539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:40.586539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:40.586539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:40.586539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:40.586539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:40.586539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:40.586539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:40.586539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:40.586539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:40.586539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:40.586539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:40.586539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 134/275] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:40.591328 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c9715-07bf-4b43-af79-40bb883162fa None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:40.615644 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2e278905-282c-4961-a21d-6de6870a8ad4 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:40.616831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2e278905-282c-4961-a21d-6de6870a8ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:40.642951 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:40.697509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 142/276] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:40.702774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e278905-282c-4961-a21d-6de6870a8ad4 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:40.733266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c9715-07bf-4b43-af79-40bb883162fa None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:40.779208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df011c1-923f-4466-9cd3-451b75c560c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:40.780322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df011c1-923f-4466-9cd3-451b75c560c4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:40.780603 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df011c1-923f-4466-9cd3-451b75c560c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:40.780876 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df011c1-923f-4466-9cd3-451b75c560c4 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:40.781590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df011c1-923f-4466-9cd3-451b75c560c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:40.781927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df011c1-923f-4466-9cd3-451b75c560c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:40.783708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df011c1-923f-4466-9cd3-451b75c560c4 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:40.794752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 135/277] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:40.800663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df011c1-923f-4466-9cd3-451b75c560c4 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:40.826148 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e278905-282c-4961-a21d-6de6870a8ad4 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:40.831076 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:40.881986 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c183594-4f5d-4dc3-8cea-2d8f69e3ec88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:40.883184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c183594-4f5d-4dc3-8cea-2d8f69e3ec88 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:40.883666 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c183594-4f5d-4dc3-8cea-2d8f69e3ec88 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:40.884398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c183594-4f5d-4dc3-8cea-2d8f69e3ec88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:40.886485 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c183594-4f5d-4dc3-8cea-2d8f69e3ec88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:40.888902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c183594-4f5d-4dc3-8cea-2d8f69e3ec88 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:40.891923 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 143/278] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:40.896980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c183594-4f5d-4dc3-8cea-2d8f69e3ec88 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:40.925357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:40.951792 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df011c1-923f-4466-9cd3-451b75c560c4 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:40.995600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1562e9-42b0-48a9-91a7-6e69b782c543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:40.996286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1562e9-42b0-48a9-91a7-6e69b782c543 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:40.996594 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1562e9-42b0-48a9-91a7-6e69b782c543 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:40.997093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1562e9-42b0-48a9-91a7-6e69b782c543 None admin] PATH_INFO: `/v3/users/e8c872e4d5bd4acd8b9b404bed7ed66f` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:41.006038 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c183594-4f5d-4dc3-8cea-2d8f69e3ec88 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:41.010990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1562e9-42b0-48a9-91a7-6e69b782c543 None admin] RBAC: Authorizing `identity:get_user(user_id=e8c872e4d5bd4acd8b9b404bed7ed66f)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:41.012429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1562e9-42b0-48a9-91a7-6e69b782c543 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:41.014597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 136/279] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:40 2022] GET /identity/v3/users/e8c872e4d5bd4acd8b9b404bed7ed66f => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:41.018984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1562e9-42b0-48a9-91a7-6e69b782c543 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:41.041245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de0fae-332b-45e0-9e9b-5c84b0708514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:41.041780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de0fae-332b-45e0-9e9b-5c84b0708514 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:41.042078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de0fae-332b-45e0-9e9b-5c84b0708514 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:41.042345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de0fae-332b-45e0-9e9b-5c84b0708514 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:41.043037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de0fae-332b-45e0-9e9b-5c84b0708514 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:41.044479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de0fae-332b-45e0-9e9b-5c84b0708514 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:41.098842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:41.119123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1562e9-42b0-48a9-91a7-6e69b782c543 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:41.136819 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 144/280] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:33:40 2022] POST /identity/v3/users => generated 293 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:33:41.137941 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de0fae-332b-45e0-9e9b-5c84b0708514 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:41.163162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c01d7-1d3a-4ead-a5ef-eea308ab06e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:41.163998 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c01d7-1d3a-4ead-a5ef-eea308ab06e7 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:41.164377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c01d7-1d3a-4ead-a5ef-eea308ab06e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:41.164792 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c01d7-1d3a-4ead-a5ef-eea308ab06e7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:41.179998 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c01d7-1d3a-4ead-a5ef-eea308ab06e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-404c01d7-1d3a-4ead-a5ef-eea308ab06e7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:41.187263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:41.188261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:41.194796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:41.194796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:41.194796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:41.194796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:41.194796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:41.194796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:41.194796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:41.194796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:41.194796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:41.194796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:41.194796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:41.194796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:41.194796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 137/281] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:41.199029 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c01d7-1d3a-4ead-a5ef-eea308ab06e7 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:41.246940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:41.268268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de0fae-332b-45e0-9e9b-5c84b0708514 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:41.306272 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e87d0-4ed9-4bd5-ad4c-4429dcadfe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:41.307415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e87d0-4ed9-4bd5-ad4c-4429dcadfe0b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:41.308493 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e87d0-4ed9-4bd5-ad4c-4429dcadfe0b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:41.309494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605e87d0-4ed9-4bd5-ad4c-4429dcadfe0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:41.310940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e87d0-4ed9-4bd5-ad4c-4429dcadfe0b None admin] RBAC: Adding query filter params (role.id=efa4ca02b2ff4ad6b14d501d4d79bbea, user.id=e8c872e4d5bd4acd8b9b404bed7ed66f, scope.system=all) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:41.311393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e87d0-4ed9-4bd5-ad4c-4429dcadfe0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:41.312862 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605e87d0-4ed9-4bd5-ad4c-4429dcadfe0b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:41.324209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 145/282] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Sep 14 19:33:41 2022] GET /identity/v3/role_assignments?user.id=e8c872e4d5bd4acd8b9b404bed7ed66f&scope.system=all&role.id=efa4ca02b2ff4ad6b14d501d4d79bbea => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:41.359177 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c01d7-1d3a-4ead-a5ef-eea308ab06e7 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:41.401169 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbeed7-0789-4724-ad73-4e1b66cabe3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:41.401787 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbeed7-0789-4724-ad73-4e1b66cabe3b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:41.402146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbeed7-0789-4724-ad73-4e1b66cabe3b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:41.402358 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abbeed7-0789-4724-ad73-4e1b66cabe3b None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:41.403031 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abbeed7-0789-4724-ad73-4e1b66cabe3b None admin] RBAC: Adding query filter params (name=reader) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:41.403334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abbeed7-0789-4724-ad73-4e1b66cabe3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:41.404831 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abbeed7-0789-4724-ad73-4e1b66cabe3b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:41.411684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:41.415385 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 138/283] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:41.421866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e87d0-4ed9-4bd5-ad4c-4429dcadfe0b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:41.534858 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605e87d0-4ed9-4bd5-ad4c-4429dcadfe0b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:41.572458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480144b-c105-46ab-b32d-54ccba0593fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:41.573014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480144b-c105-46ab-b32d-54ccba0593fb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:41.573290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480144b-c105-46ab-b32d-54ccba0593fb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:41.573558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2480144b-c105-46ab-b32d-54ccba0593fb None admin] PATH_INFO: `/v3/users/63f30ef3af5e4c4dbdadc233888e3ccc` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:41.574717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480144b-c105-46ab-b32d-54ccba0593fb None admin] RBAC: Authorizing `identity:get_user(user_id=63f30ef3af5e4c4dbdadc233888e3ccc)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:41.576162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2480144b-c105-46ab-b32d-54ccba0593fb None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:41.581567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 146/284] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:41 2022] GET /identity/v3/users/63f30ef3af5e4c4dbdadc233888e3ccc => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:41.584011 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbeed7-0789-4724-ad73-4e1b66cabe3b None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:41.696316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abbeed7-0789-4724-ad73-4e1b66cabe3b None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:41.733142 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85442e6-cb09-45e5-9a2f-25f0542c61ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:41.734174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85442e6-cb09-45e5-9a2f-25f0542c61ab None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:41.734742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85442e6-cb09-45e5-9a2f-25f0542c61ab None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:41.735212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85442e6-cb09-45e5-9a2f-25f0542c61ab None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:41.736551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85442e6-cb09-45e5-9a2f-25f0542c61ab None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:41.738224 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85442e6-cb09-45e5-9a2f-25f0542c61ab None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:41.744940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 139/285] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:41 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:41.746558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480144b-c105-46ab-b32d-54ccba0593fb None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:41.763654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:41.864072 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2480144b-c105-46ab-b32d-54ccba0593fb None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:41.900156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edc7406-ab94-4423-a3a2-dc129b7e9eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:41.901172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc7406-ab94-4423-a3a2-dc129b7e9eb7 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:41.901663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc7406-ab94-4423-a3a2-dc129b7e9eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:41.902153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edc7406-ab94-4423-a3a2-dc129b7e9eb7 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552/users/63f30ef3af5e4c4dbdadc233888e3ccc/roles/35aa74738f2d4844af99831724f20d39` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:41.904229 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc7406-ab94-4423-a3a2-dc129b7e9eb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5fbdcb3c4884d59ab798fc1c9752552, user_id=63f30ef3af5e4c4dbdadc233888e3ccc, role_id=35aa74738f2d4844af99831724f20d39)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:41.905883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edc7406-ab94-4423-a3a2-dc129b7e9eb7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:41.989888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 147/286] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:33:41 2022] PUT /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552/users/63f30ef3af5e4c4dbdadc233888e3ccc/roles/35aa74738f2d4844af99831724f20d39 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:42.345766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bac2fb-f982-4f37-8b86-510a1c415514 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:42.346296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bac2fb-f982-4f37-8b86-510a1c415514 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:42.346396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bac2fb-f982-4f37-8b86-510a1c415514 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:42.347907 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 140/287] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:42.353566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819af897-9e4e-4aad-bc28-3aa3d835686c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:42.354082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819af897-9e4e-4aad-bc28-3aa3d835686c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:42.354159 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819af897-9e4e-4aad-bc28-3aa3d835686c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:42.376903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-819af897-9e4e-4aad-bc28-3aa3d835686c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:42.378131 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-819af897-9e4e-4aad-bc28-3aa3d835686c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:42.456377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 148/288] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:42.947723 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bac2fb-f982-4f37-8b86-510a1c415514 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:43.027047 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:43.074245 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bac2fb-f982-4f37-8b86-510a1c415514 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:43.120283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd0551-dcda-4938-a2a6-465b92eed98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:43.120962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd0551-dcda-4938-a2a6-465b92eed98d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:43.121239 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd0551-dcda-4938-a2a6-465b92eed98d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:43.121503 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd0551-dcda-4938-a2a6-465b92eed98d None admin] PATH_INFO: `/v3/roles/member` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:43.138746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd0551-dcda-4938-a2a6-465b92eed98d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-17fd0551-dcda-4938-a2a6-465b92eed98d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:43.144579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:43.145524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:43.146378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:43.146378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:43.146378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:43.146378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:43.146378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:43.146378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:43.146378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:43.146378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:43.146378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:43.146378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:43.146378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:43.146378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:43.148437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 141/289] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:43.155452 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819af897-9e4e-4aad-bc28-3aa3d835686c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:43.156168 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:43.290435 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819af897-9e4e-4aad-bc28-3aa3d835686c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:43.345638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d98e09-f433-43d5-9650-1933502cbde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:43.346676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d98e09-f433-43d5-9650-1933502cbde7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:43.347158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d98e09-f433-43d5-9650-1933502cbde7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:43.347631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d98e09-f433-43d5-9650-1933502cbde7 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:43.348495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d98e09-f433-43d5-9650-1933502cbde7 None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:43.348996 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d98e09-f433-43d5-9650-1933502cbde7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:43.350604 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d98e09-f433-43d5-9650-1933502cbde7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:43.359994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 149/290] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:43.366498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd0551-dcda-4938-a2a6-465b92eed98d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:43.511737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd0551-dcda-4938-a2a6-465b92eed98d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:43.548315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:43.557310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa56edc-e81c-435e-ab88-e40c4ae05ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:43.557978 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa56edc-e81c-435e-ab88-e40c4ae05ab3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:43.558251 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa56edc-e81c-435e-ab88-e40c4ae05ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:43.558523 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa56edc-e81c-435e-ab88-e40c4ae05ab3 None admin] PATH_INFO: `/v3/users/b9e128fc07ef420e94e299b50d370327` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:43.562487 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa56edc-e81c-435e-ab88-e40c4ae05ab3 None admin] RBAC: Authorizing `identity:get_user(user_id=b9e128fc07ef420e94e299b50d370327)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:43.566846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa56edc-e81c-435e-ab88-e40c4ae05ab3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:43.566846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 142/291] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:43 2022] GET /identity/v3/users/b9e128fc07ef420e94e299b50d370327 => generated 297 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:43.571893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d98e09-f433-43d5-9650-1933502cbde7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:43.746899 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:43.779108 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d3c53-3bcf-4a1b-adac-73711d29b213 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:43.779897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d3c53-3bcf-4a1b-adac-73711d29b213 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:43.780099 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d3c53-3bcf-4a1b-adac-73711d29b213 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:43.781704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 143/292] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:33:43.786657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477bead-7ab5-47f2-bfa8-6de4f4eb2ed7 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:43.786934 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477bead-7ab5-47f2-bfa8-6de4f4eb2ed7 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:43.787201 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8477bead-7ab5-47f2-bfa8-6de4f4eb2ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:43.805011 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-8477bead-7ab5-47f2-bfa8-6de4f4eb2ed7 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:43.806238 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8477bead-7ab5-47f2-bfa8-6de4f4eb2ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:43.817822 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d98e09-f433-43d5-9650-1933502cbde7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:43.877826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfa0488-03e5-4449-9d07-681af0ca483d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:43.877826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa0488-03e5-4449-9d07-681af0ca483d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:43.877826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa0488-03e5-4449-9d07-681af0ca483d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:43.877826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfa0488-03e5-4449-9d07-681af0ca483d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:43.914252 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 144/293] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:43.990752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa0488-03e5-4449-9d07-681af0ca483d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:43.992641 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfa0488-03e5-4449-9d07-681af0ca483d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:44.048165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ff6d4-aa68-4785-ab29-f0f3d96c78a7 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:44.048478 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ff6d4-aa68-4785-ab29-f0f3d96c78a7 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:44.048714 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ff6d4-aa68-4785-ab29-f0f3d96c78a7 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:44.050457 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 145/294] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:44.055167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f39d11-9c3b-461d-98e5-57efcd04f7b6 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:44.055449 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f39d11-9c3b-461d-98e5-57efcd04f7b6 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:44.055718 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f39d11-9c3b-461d-98e5-57efcd04f7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:44.082780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:44.093936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e4f39d11-9c3b-461d-98e5-57efcd04f7b6 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:44.096217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e4f39d11-9c3b-461d-98e5-57efcd04f7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:44.132681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 150/295] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:44.140844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec216ddb-61ea-4520-a2b5-5922203176c3 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:44.141183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec216ddb-61ea-4520-a2b5-5922203176c3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:44.141390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec216ddb-61ea-4520-a2b5-5922203176c3 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:44.142928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 151/296] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:44.143632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec216ddb-61ea-4520-a2b5-5922203176c3 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:44.188881 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 146/297] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:44.190634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd83256-6793-4884-840e-12499def616b None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:44.191123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd83256-6793-4884-840e-12499def616b None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:44.191613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd83256-6793-4884-840e-12499def616b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:44.211537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-1bd83256-6793-4884-840e-12499def616b None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:44.213637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1bd83256-6793-4884-840e-12499def616b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:44.274935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec216ddb-61ea-4520-a2b5-5922203176c3 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:44.286929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:44.300682 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 147/298] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:44.301945 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd83256-6793-4884-840e-12499def616b None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:44.315051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10913f4f-efd8-4f25-9f5b-cceaa8f314f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:44.316200 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10913f4f-efd8-4f25-9f5b-cceaa8f314f7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:44.316911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10913f4f-efd8-4f25-9f5b-cceaa8f314f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:44.317721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10913f4f-efd8-4f25-9f5b-cceaa8f314f7 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:44.319015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10913f4f-efd8-4f25-9f5b-cceaa8f314f7 None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:44.320620 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10913f4f-efd8-4f25-9f5b-cceaa8f314f7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:44.323191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 152/299] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:44 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:44.329852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10913f4f-efd8-4f25-9f5b-cceaa8f314f7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:44.389519 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:44.476861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd83256-6793-4884-840e-12499def616b None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:44.517526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10913f4f-efd8-4f25-9f5b-cceaa8f314f7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:44.531704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ebb55-bdf3-476b-88e0-e4c65a5fd61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:44.532346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ebb55-bdf3-476b-88e0-e4c65a5fd61e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:44.532625 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ebb55-bdf3-476b-88e0-e4c65a5fd61e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:44.532897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ebb55-bdf3-476b-88e0-e4c65a5fd61e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:44.543524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ebb55-bdf3-476b-88e0-e4c65a5fd61e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-ff0ebb55-bdf3-476b-88e0-e4c65a5fd61e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:44.548650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:44.549519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:44.550364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:44.550364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:44.550364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:44.550364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:44.550364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:44.550364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:44.550364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:44.550364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:44.550364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:44.550364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:44.550364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:44.550364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:44.550364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 148/300] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:44.551059 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ebb55-bdf3-476b-88e0-e4c65a5fd61e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:44.562228 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5983060-7fab-4c66-8b91-dbdbae01c106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:44.563189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5983060-7fab-4c66-8b91-dbdbae01c106 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:44.563667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5983060-7fab-4c66-8b91-dbdbae01c106 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:44.564246 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5983060-7fab-4c66-8b91-dbdbae01c106 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:44.565159 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5983060-7fab-4c66-8b91-dbdbae01c106 None admin] RBAC: Adding query filter params (role.id=efa4ca02b2ff4ad6b14d501d4d79bbea, scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552, user.id=b9e128fc07ef420e94e299b50d370327) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:44.565759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5983060-7fab-4c66-8b91-dbdbae01c106 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:44.568057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5983060-7fab-4c66-8b91-dbdbae01c106 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:44.582138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 153/301] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:44 2022] GET /identity/v3/role_assignments?user.id=b9e128fc07ef420e94e299b50d370327&scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552&role.id=efa4ca02b2ff4ad6b14d501d4d79bbea => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:44.582947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5983060-7fab-4c66-8b91-dbdbae01c106 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:44.628819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:44.689272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ebb55-bdf3-476b-88e0-e4c65a5fd61e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:44.718784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:44.726062 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5983060-7fab-4c66-8b91-dbdbae01c106 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:44.730703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea7910-9163-40d8-9844-436385b660c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:44.731346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea7910-9163-40d8-9844-436385b660c1 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:44.731622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea7910-9163-40d8-9844-436385b660c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:44.731891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea7910-9163-40d8-9844-436385b660c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:44.743967 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caea7910-9163-40d8-9844-436385b660c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-caea7910-9163-40d8-9844-436385b660c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:44.749549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:44.750705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:44.751655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:44.751655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:44.751655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:44.751655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:44.751655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:44.751655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:44.751655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:44.751655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:44.751655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:44.751655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:44.751655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:44.751655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:44.751655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 149/302] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:44.752606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea7910-9163-40d8-9844-436385b660c1 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:44.772690 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9273e-a941-491a-85f0-0b371db66c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:44.773888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9273e-a941-491a-85f0-0b371db66c5b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:44.774752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9273e-a941-491a-85f0-0b371db66c5b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:44.775198 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9273e-a941-491a-85f0-0b371db66c5b None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:44.776038 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9273e-a941-491a-85f0-0b371db66c5b None admin] RBAC: Adding query filter params (name=reader) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:44.776519 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9273e-a941-491a-85f0-0b371db66c5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:44.778400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9273e-a941-491a-85f0-0b371db66c5b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:44.789887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 154/303] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:44.791058 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9273e-a941-491a-85f0-0b371db66c5b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:44.904212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea7910-9163-40d8-9844-436385b660c1 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:44.912806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:44.924026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9273e-a941-491a-85f0-0b371db66c5b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:44.940623 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7e07f-bb04-46f8-b8d8-1701255ec862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:44.941339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7e07f-bb04-46f8-b8d8-1701255ec862 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:44.941621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7e07f-bb04-46f8-b8d8-1701255ec862 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:44.941915 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7e07f-bb04-46f8-b8d8-1701255ec862 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:44.943264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7e07f-bb04-46f8-b8d8-1701255ec862 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:44.944693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7e07f-bb04-46f8-b8d8-1701255ec862 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:44.950781 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 150/304] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:44.952643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7e07f-bb04-46f8-b8d8-1701255ec862 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:44.961123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b1df1-6dec-400c-8830-bdbdc32a3a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:44.961733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b1df1-6dec-400c-8830-bdbdc32a3a2f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:44.962037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b1df1-6dec-400c-8830-bdbdc32a3a2f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:44.962335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b1df1-6dec-400c-8830-bdbdc32a3a2f None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:44.962990 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b1df1-6dec-400c-8830-bdbdc32a3a2f None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:44.963294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b1df1-6dec-400c-8830-bdbdc32a3a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:44.964720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b1df1-6dec-400c-8830-bdbdc32a3a2f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:44.973200 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 155/305] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:44.974000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b1df1-6dec-400c-8830-bdbdc32a3a2f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:44.996932 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:45.050412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7e07f-bb04-46f8-b8d8-1701255ec862 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:45.074251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b1df1-6dec-400c-8830-bdbdc32a3a2f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:45.083704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b0a06-8e1b-4c44-8dc9-83f07ef29551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:45.084332 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b0a06-8e1b-4c44-8dc9-83f07ef29551 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:45.084612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b0a06-8e1b-4c44-8dc9-83f07ef29551 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:45.084884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b0a06-8e1b-4c44-8dc9-83f07ef29551 None admin] PATH_INFO: `/v3/users/0e90add7e5a248a9b6d8de822459a1f7` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:45.097450 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b0a06-8e1b-4c44-8dc9-83f07ef29551 None admin] RBAC: Authorizing `identity:get_user(user_id=0e90add7e5a248a9b6d8de822459a1f7)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:45.098923 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474b0a06-8e1b-4c44-8dc9-83f07ef29551 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:45.100498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 151/306] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:44 2022] GET /identity/v3/users/0e90add7e5a248a9b6d8de822459a1f7 => generated 293 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:45.101626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff630d-15c4-4574-a3db-252b25ac1598 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:45.101931 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff630d-15c4-4574-a3db-252b25ac1598 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:45.102206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff630d-15c4-4574-a3db-252b25ac1598 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:45.103397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 152/307] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:45.104091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff630d-15c4-4574-a3db-252b25ac1598 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:45.111636 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9a927-416f-4532-ad3d-bf53fe0a5bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:45.112243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a927-416f-4532-ad3d-bf53fe0a5bd5 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:45.112517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a927-416f-4532-ad3d-bf53fe0a5bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:45.112806 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a927-416f-4532-ad3d-bf53fe0a5bd5 None admin] PATH_INFO: `/v3/groups` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:45.113421 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9a927-416f-4532-ad3d-bf53fe0a5bd5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:45.114972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9a927-416f-4532-ad3d-bf53fe0a5bd5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:45.148543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:45.200533 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff630d-15c4-4574-a3db-252b25ac1598 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:45.225708 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 156/308] 10.222.0.42 () {60 vars in 1134 bytes} [Wed Sep 14 19:33:44 2022] POST /identity/v3/groups => generated 229 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:33:45.226547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9a927-416f-4532-ad3d-bf53fe0a5bd5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:45.237731 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd66a42-b97c-4b2b-98e4-1ff034e14169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:45.238423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd66a42-b97c-4b2b-98e4-1ff034e14169 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:45.238696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd66a42-b97c-4b2b-98e4-1ff034e14169 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:45.238963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd66a42-b97c-4b2b-98e4-1ff034e14169 None admin] PATH_INFO: `/v3/users/e8c872e4d5bd4acd8b9b404bed7ed66f` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:45.240182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd66a42-b97c-4b2b-98e4-1ff034e14169 None admin] RBAC: Authorizing `identity:get_user(user_id=e8c872e4d5bd4acd8b9b404bed7ed66f)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:45.241713 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd66a42-b97c-4b2b-98e4-1ff034e14169 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:45.248299 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 153/309] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:45 2022] GET /identity/v3/users/e8c872e4d5bd4acd8b9b404bed7ed66f => generated 293 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:45.248299 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9301a-10dc-4298-acd3-36b8cc3c5dc9 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:45.248299 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9301a-10dc-4298-acd3-36b8cc3c5dc9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:45.248299 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9301a-10dc-4298-acd3-36b8cc3c5dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:45.264125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-45a9301a-10dc-4298-acd3-36b8cc3c5dc9 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:45.265379 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-45a9301a-10dc-4298-acd3-36b8cc3c5dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:45.331176 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:45.340655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 154/310] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:45.341418 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9301a-10dc-4298-acd3-36b8cc3c5dc9 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:45.362608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9a927-416f-4532-ad3d-bf53fe0a5bd5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:45.378448 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:45.416853 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243a4d7-2357-4b3f-bfa6-76bf31a45515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:45.418380 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a4d7-2357-4b3f-bfa6-76bf31a45515 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:45.419381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a4d7-2357-4b3f-bfa6-76bf31a45515 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:45.419656 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a4d7-2357-4b3f-bfa6-76bf31a45515 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:45.420897 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243a4d7-2357-4b3f-bfa6-76bf31a45515 None admin] RBAC: Adding query filter params (role.id=35aa74738f2d4844af99831724f20d39, user.id=0e90add7e5a248a9b6d8de822459a1f7, scope.system=all) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:45.421209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243a4d7-2357-4b3f-bfa6-76bf31a45515 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:45.423924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243a4d7-2357-4b3f-bfa6-76bf31a45515 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:45.430970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 157/311] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Sep 14 19:33:45 2022] GET /identity/v3/role_assignments?user.id=0e90add7e5a248a9b6d8de822459a1f7&scope.system=all&role.id=35aa74738f2d4844af99831724f20d39 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:45.458808 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9301a-10dc-4298-acd3-36b8cc3c5dc9 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:45.498408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7fdc3-466c-48cc-9ceb-f43f20ce4ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:45.499162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7fdc3-466c-48cc-9ceb-f43f20ce4ecd None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:45.499350 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7fdc3-466c-48cc-9ceb-f43f20ce4ecd None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:45.499623 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7fdc3-466c-48cc-9ceb-f43f20ce4ecd None admin] PATH_INFO: `/v3/system/users/e8c872e4d5bd4acd8b9b404bed7ed66f/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:45.501097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7fdc3-466c-48cc-9ceb-f43f20ce4ecd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e8c872e4d5bd4acd8b9b404bed7ed66f, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:45.502565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7fdc3-466c-48cc-9ceb-f43f20ce4ecd None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:45.617752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243a4d7-2357-4b3f-bfa6-76bf31a45515 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:45.623655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 155/312] 10.222.0.42 () {58 vars in 1334 bytes} [Wed Sep 14 19:33:45 2022] PUT /identity/v3/system/users/e8c872e4d5bd4acd8b9b404bed7ed66f/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:45.730164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243a4d7-2357-4b3f-bfa6-76bf31a45515 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:45.767107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcccfa9-80ae-4828-a139-ea93de0ed5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:45.767812 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcccfa9-80ae-4828-a139-ea93de0ed5eb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:45.768083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcccfa9-80ae-4828-a139-ea93de0ed5eb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:45.768398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcccfa9-80ae-4828-a139-ea93de0ed5eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:45.781166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcccfa9-80ae-4828-a139-ea93de0ed5eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-1dcccfa9-80ae-4828-a139-ea93de0ed5eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:45.787575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:45.788683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:45.789750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:45.789750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:45.789750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:45.789750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:45.789750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:45.789750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:45.789750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:45.789750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:45.789750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:45.789750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:45.789750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:45.789750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:45.789750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 158/313] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:33:45.798970 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7fdc3-466c-48cc-9ceb-f43f20ce4ecd None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:45.914379 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:45.922437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7fdc3-466c-48cc-9ceb-f43f20ce4ecd None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:45.960223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338065c5-879f-40c1-8632-be220a128580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:45.960832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338065c5-879f-40c1-8632-be220a128580 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:45.961107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338065c5-879f-40c1-8632-be220a128580 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:45.961373 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338065c5-879f-40c1-8632-be220a128580 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:45.965840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338065c5-879f-40c1-8632-be220a128580 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:45.965840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338065c5-879f-40c1-8632-be220a128580 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:45.965840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338065c5-879f-40c1-8632-be220a128580 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:45.974371 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 156/314] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:45.984373 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcccfa9-80ae-4828-a139-ea93de0ed5eb None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:45.988881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:46.087442 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcccfa9-80ae-4828-a139-ea93de0ed5eb None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:46.120815 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad54a7c-c14a-445f-84cc-6f5583c8676d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:46.121447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad54a7c-c14a-445f-84cc-6f5583c8676d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:46.121725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad54a7c-c14a-445f-84cc-6f5583c8676d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:46.122017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad54a7c-c14a-445f-84cc-6f5583c8676d None admin] PATH_INFO: `/v3/users/63f30ef3af5e4c4dbdadc233888e3ccc` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:46.123671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad54a7c-c14a-445f-84cc-6f5583c8676d None admin] RBAC: Authorizing `identity:get_user(user_id=63f30ef3af5e4c4dbdadc233888e3ccc)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:46.125118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad54a7c-c14a-445f-84cc-6f5583c8676d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:46.127075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 159/315] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:45 2022] GET /identity/v3/users/63f30ef3af5e4c4dbdadc233888e3ccc => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:46.134905 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338065c5-879f-40c1-8632-be220a128580 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:46.243053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338065c5-879f-40c1-8632-be220a128580 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:46.276352 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece5202-808f-4442-b821-418879341f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:46.276959 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece5202-808f-4442-b821-418879341f89 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:46.277231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece5202-808f-4442-b821-418879341f89 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:46.277495 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece5202-808f-4442-b821-418879341f89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:46.305424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:46.347847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece5202-808f-4442-b821-418879341f89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:46.349311 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece5202-808f-4442-b821-418879341f89 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:46.453026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 157/316] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:46.458867 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad54a7c-c14a-445f-84cc-6f5583c8676d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:46.528918 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:46.558681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad54a7c-c14a-445f-84cc-6f5583c8676d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:46.601910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca59fc6-bbb8-4d18-a6e3-34ba6f63d495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:46.602585 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca59fc6-bbb8-4d18-a6e3-34ba6f63d495 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:46.602859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca59fc6-bbb8-4d18-a6e3-34ba6f63d495 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:46.603125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca59fc6-bbb8-4d18-a6e3-34ba6f63d495 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:46.607179 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca59fc6-bbb8-4d18-a6e3-34ba6f63d495 None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:46.609168 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca59fc6-bbb8-4d18-a6e3-34ba6f63d495 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:46.614073 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 160/317] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:46 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:46.622907 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece5202-808f-4442-b821-418879341f89 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:46.680492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:46.726441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece5202-808f-4442-b821-418879341f89 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:46.761704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f83a4c-a654-4f22-8e2a-16acc0310f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:46.762324 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f83a4c-a654-4f22-8e2a-16acc0310f66 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:46.762652 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f83a4c-a654-4f22-8e2a-16acc0310f66 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:46.762920 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f83a4c-a654-4f22-8e2a-16acc0310f66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:46.763702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f83a4c-a654-4f22-8e2a-16acc0310f66 None admin] RBAC: Adding query filter params (role.id=35aa74738f2d4844af99831724f20d39, scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552, user.id=63f30ef3af5e4c4dbdadc233888e3ccc) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:46.764006 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f83a4c-a654-4f22-8e2a-16acc0310f66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:46.765481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f83a4c-a654-4f22-8e2a-16acc0310f66 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:46.771590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 158/318] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:33:46 2022] GET /identity/v3/role_assignments?user.id=63f30ef3af5e4c4dbdadc233888e3ccc&scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552&role.id=35aa74738f2d4844af99831724f20d39 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:47.482800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d2d82-3fe0-4244-bc1d-60cf9e05a8be None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:47.483340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d2d82-3fe0-4244-bc1d-60cf9e05a8be None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:47.483340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d2d82-3fe0-4244-bc1d-60cf9e05a8be None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:47.484712 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 161/319] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:47.489862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c92e5-c962-49b2-9be2-d0767166f6f5 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:47.490163 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c92e5-c962-49b2-9be2-d0767166f6f5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:47.495781 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639c92e5-c962-49b2-9be2-d0767166f6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:47.508842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-639c92e5-c962-49b2-9be2-d0767166f6f5 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:47.510076 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-639c92e5-c962-49b2-9be2-d0767166f6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:47.631736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 159/320] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:47.731226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31e47a-e067-4fc8-a044-9a17b26c8719 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:47.731579 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31e47a-e067-4fc8-a044-9a17b26c8719 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:47.731781 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f31e47a-e067-4fc8-a044-9a17b26c8719 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:47.733328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 162/321] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:47.738302 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9064ce-965d-4351-8182-68c86cdab0b4 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:47.738584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9064ce-965d-4351-8182-68c86cdab0b4 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:47.738851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9064ce-965d-4351-8182-68c86cdab0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:47.758000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-bf9064ce-965d-4351-8182-68c86cdab0b4 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:47.759223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bf9064ce-965d-4351-8182-68c86cdab0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:47.808218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:47.829248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 160/322] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:47.875602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313238c1-2253-4ce7-9223-90c1efec9f08 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:47.875883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313238c1-2253-4ce7-9223-90c1efec9f08 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:47.876150 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313238c1-2253-4ce7-9223-90c1efec9f08 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:47.877632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 163/323] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:47.882636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcf1ce-c496-4c4c-a219-607373f0a465 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:47.882928 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcf1ce-c496-4c4c-a219-607373f0a465 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:47.883205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcf1ce-c496-4c4c-a219-607373f0a465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:47.897672 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313238c1-2253-4ce7-9223-90c1efec9f08 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:47.901316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-5cbcf1ce-c496-4c4c-a219-607373f0a465 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:47.902525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-5cbcf1ce-c496-4c4c-a219-607373f0a465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:47.982203 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 161/324] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:48.013112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313238c1-2253-4ce7-9223-90c1efec9f08 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:48.047026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213494b-6a46-4918-bcc1-71c8a2140c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:48.047653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213494b-6a46-4918-bcc1-71c8a2140c5a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:48.047931 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213494b-6a46-4918-bcc1-71c8a2140c5a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:48.048201 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213494b-6a46-4918-bcc1-71c8a2140c5a None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:48.049282 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213494b-6a46-4918-bcc1-71c8a2140c5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:48.050742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213494b-6a46-4918-bcc1-71c8a2140c5a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:48.052522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 164/325] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:33:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:48.057903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbcf1ce-c496-4c4c-a219-607373f0a465 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:48.098087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213494b-6a46-4918-bcc1-71c8a2140c5a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:48.111837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:48.163669 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbcf1ce-c496-4c4c-a219-607373f0a465 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:48.175923 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:48.199763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418b918-1346-4d51-a979-d6fa9f67f402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:48.200374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418b918-1346-4d51-a979-d6fa9f67f402 None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:48.200656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418b918-1346-4d51-a979-d6fa9f67f402 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:48.200926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418b918-1346-4d51-a979-d6fa9f67f402 None admin] PATH_INFO: `/v3/groups` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:48.201619 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418b918-1346-4d51-a979-d6fa9f67f402 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:48.203423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418b918-1346-4d51-a979-d6fa9f67f402 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:48.208583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213494b-6a46-4918-bcc1-71c8a2140c5a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:48.242758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33b3a4-47aa-4d1c-9707-449a1f4181e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:48.243390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33b3a4-47aa-4d1c-9707-449a1f4181e3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:48.243664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33b3a4-47aa-4d1c-9707-449a1f4181e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:48.243935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33b3a4-47aa-4d1c-9707-449a1f4181e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:48.254256 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33b3a4-47aa-4d1c-9707-449a1f4181e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-7d33b3a4-47aa-4d1c-9707-449a1f4181e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:48.259447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:48.260432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:48.261261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:48.261261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:48.261261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:48.261261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:48.261261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:48.261261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:48.261261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:48.261261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:48.261261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:48.261261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:48.261261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:48.261261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:48.261261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 165/326] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:33:48.261934 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33b3a4-47aa-4d1c-9707-449a1f4181e3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:48.300032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 162/327] 10.222.0.42 () {60 vars in 1134 bytes} [Wed Sep 14 19:33:48 2022] POST /identity/v3/groups => generated 226 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:33:48.304676 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418b918-1346-4d51-a979-d6fa9f67f402 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:48.337353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:48.375642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33b3a4-47aa-4d1c-9707-449a1f4181e3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:48.412786 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42bd8fd-0651-4ae1-97f3-03a071c5c2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:48.413916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bd8fd-0651-4ae1-97f3-03a071c5c2b7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:48.414218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bd8fd-0651-4ae1-97f3-03a071c5c2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:48.414666 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42bd8fd-0651-4ae1-97f3-03a071c5c2b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:48.416500 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418b918-1346-4d51-a979-d6fa9f67f402 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:48.426034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42bd8fd-0651-4ae1-97f3-03a071c5c2b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-e42bd8fd-0651-4ae1-97f3-03a071c5c2b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:48.432356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:48.433358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:48.434207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:48.434207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:48.434207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:48.434207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:48.434207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:48.434207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:48.434207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:48.434207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:48.434207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:48.434207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:48.434207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:48.434207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:48.434207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 166/328] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:48.442660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42bd8fd-0651-4ae1-97f3-03a071c5c2b7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:48.460043 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811156f3-2704-4f75-85d0-75564e939308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:48.460668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811156f3-2704-4f75-85d0-75564e939308 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:48.460947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811156f3-2704-4f75-85d0-75564e939308 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:48.461217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811156f3-2704-4f75-85d0-75564e939308 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:48.461911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811156f3-2704-4f75-85d0-75564e939308 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:48.462212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811156f3-2704-4f75-85d0-75564e939308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:48.463604 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811156f3-2704-4f75-85d0-75564e939308 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:48.474962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 163/329] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:48.478371 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811156f3-2704-4f75-85d0-75564e939308 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:48.514751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:48.604140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42bd8fd-0651-4ae1-97f3-03a071c5c2b7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:48.634054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:48.643608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811156f3-2704-4f75-85d0-75564e939308 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:48.652104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fd892-b411-4cce-b27c-d43abe8c8409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:48.653003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fd892-b411-4cce-b27c-d43abe8c8409 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:48.653506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fd892-b411-4cce-b27c-d43abe8c8409 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:48.654068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788fd892-b411-4cce-b27c-d43abe8c8409 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:48.655466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fd892-b411-4cce-b27c-d43abe8c8409 None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:48.655966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fd892-b411-4cce-b27c-d43abe8c8409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:48.659358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788fd892-b411-4cce-b27c-d43abe8c8409 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:48.669346 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 167/330] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:48.675128 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fd892-b411-4cce-b27c-d43abe8c8409 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:48.685708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb3856-3501-4001-9b03-6bb61b9bde38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:48.686396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb3856-3501-4001-9b03-6bb61b9bde38 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:48.686668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb3856-3501-4001-9b03-6bb61b9bde38 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:48.686935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfb3856-3501-4001-9b03-6bb61b9bde38 None admin] PATH_INFO: `/v3/users/0e90add7e5a248a9b6d8de822459a1f7` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:48.688072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb3856-3501-4001-9b03-6bb61b9bde38 None admin] RBAC: Authorizing `identity:get_user(user_id=0e90add7e5a248a9b6d8de822459a1f7)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:48.689548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfb3856-3501-4001-9b03-6bb61b9bde38 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:48.691539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 164/331] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:48 2022] GET /identity/v3/users/0e90add7e5a248a9b6d8de822459a1f7 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:48.698058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb3856-3501-4001-9b03-6bb61b9bde38 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:48.807515 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788fd892-b411-4cce-b27c-d43abe8c8409 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:48.835365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:48.838994 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfb3856-3501-4001-9b03-6bb61b9bde38 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:48.853035 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f67b3-b312-4282-bddf-d9d95d9eb377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:48.853685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f67b3-b312-4282-bddf-d9d95d9eb377 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:48.853997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f67b3-b312-4282-bddf-d9d95d9eb377 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:48.854319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f67b3-b312-4282-bddf-d9d95d9eb377 None admin] PATH_INFO: `/v3/users/e8c872e4d5bd4acd8b9b404bed7ed66f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:48.855446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f67b3-b312-4282-bddf-d9d95d9eb377 None admin] RBAC: Authorizing `identity:get_user(user_id=e8c872e4d5bd4acd8b9b404bed7ed66f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:48.856890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213f67b3-b312-4282-bddf-d9d95d9eb377 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:48.860072 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 168/332] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:48 2022] GET /identity/v3/users/e8c872e4d5bd4acd8b9b404bed7ed66f => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:48.866192 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f67b3-b312-4282-bddf-d9d95d9eb377 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:48.889946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd72216-e34b-40b2-bb49-6e4aef680839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:48.890607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd72216-e34b-40b2-bb49-6e4aef680839 None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:48.890888 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd72216-e34b-40b2-bb49-6e4aef680839 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:48.891162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd72216-e34b-40b2-bb49-6e4aef680839 None admin] PATH_INFO: `/v3/system/users/0e90add7e5a248a9b6d8de822459a1f7/roles/35aa74738f2d4844af99831724f20d39` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:48.893368 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd72216-e34b-40b2-bb49-6e4aef680839 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0e90add7e5a248a9b6d8de822459a1f7, role_id=35aa74738f2d4844af99831724f20d39)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:48.895289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd72216-e34b-40b2-bb49-6e4aef680839 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:48.938735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:48.981494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f67b3-b312-4282-bddf-d9d95d9eb377 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:48.981678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 165/333] 10.222.0.42 () {58 vars in 1334 bytes} [Wed Sep 14 19:33:48 2022] PUT /identity/v3/system/users/0e90add7e5a248a9b6d8de822459a1f7/roles/35aa74738f2d4844af99831724f20d39 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:49.023324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b7e9e0-5f23-4340-8011-5c58280dbfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:49.024348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7e9e0-5f23-4340-8011-5c58280dbfc4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:49.024871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7e9e0-5f23-4340-8011-5c58280dbfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:49.025372 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b7e9e0-5f23-4340-8011-5c58280dbfc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:49.026384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7e9e0-5f23-4340-8011-5c58280dbfc4 None admin] RBAC: Adding query filter params (role.id=efa4ca02b2ff4ad6b14d501d4d79bbea, user.id=e8c872e4d5bd4acd8b9b404bed7ed66f, scope.system=all) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:49.026975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7e9e0-5f23-4340-8011-5c58280dbfc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:49.028690 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b7e9e0-5f23-4340-8011-5c58280dbfc4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:49.044321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 169/334] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Sep 14 19:33:48 2022] GET /identity/v3/role_assignments?user.id=e8c872e4d5bd4acd8b9b404bed7ed66f&scope.system=all&role.id=efa4ca02b2ff4ad6b14d501d4d79bbea => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:50.737090 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f344e-5873-4026-89fe-b955fc9ec5b2 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:50.737647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f344e-5873-4026-89fe-b955fc9ec5b2 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:50.737647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9f344e-5873-4026-89fe-b955fc9ec5b2 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:50.739016 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 166/335] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:50.744492 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b50401-5b12-447d-81fa-7d8e991ff68d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:50.744904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b50401-5b12-447d-81fa-7d8e991ff68d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:50.745091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b50401-5b12-447d-81fa-7d8e991ff68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:50.774810 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f9b50401-5b12-447d-81fa-7d8e991ff68d None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:50.776034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f9b50401-5b12-447d-81fa-7d8e991ff68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:50.859388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 170/336] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:51.419704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f344e-5873-4026-89fe-b955fc9ec5b2 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:51.556156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9f344e-5873-4026-89fe-b955fc9ec5b2 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:51.601568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2600e6f-8041-45cd-9ba7-a7f8b2011130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:51.602238 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2600e6f-8041-45cd-9ba7-a7f8b2011130 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:51.602519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2600e6f-8041-45cd-9ba7-a7f8b2011130 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:51.602791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2600e6f-8041-45cd-9ba7-a7f8b2011130 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:51.616115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217b55-06bb-4eca-9fc5-68d6dc78e7e7 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:51.617571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217b55-06bb-4eca-9fc5-68d6dc78e7e7 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:51.617571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35217b55-06bb-4eca-9fc5-68d6dc78e7e7 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:51.617737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2600e6f-8041-45cd-9ba7-a7f8b2011130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:51.618194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 171/337] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-f2600e6f-8041-45cd-9ba7-a7f8b2011130 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:51.622716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:51.623702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:51.624561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:51.624561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:51.624561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:51.624561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:51.624561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:51.624561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:51.624561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:51.624561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:51.624561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:51.624561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:51.624561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:51.624561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:51.624561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 167/338] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:51.626510 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7eea-7fd6-4ac1-99ed-c6e725813128 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:51.627001 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7eea-7fd6-4ac1-99ed-c6e725813128 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:51.627464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab7eea-7fd6-4ac1-99ed-c6e725813128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:51.629635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2600e6f-8041-45cd-9ba7-a7f8b2011130 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:51.648848 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c0ab7eea-7fd6-4ac1-99ed-c6e725813128 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:51.651306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c0ab7eea-7fd6-4ac1-99ed-c6e725813128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:51.661702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:51.717150 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:51.750216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 172/339] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:51.752134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2f2a7-2539-4b87-82d7-bb0cef4de894 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:51.752416 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2f2a7-2539-4b87-82d7-bb0cef4de894 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:51.752685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c2f2a7-2539-4b87-82d7-bb0cef4de894 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:51.755140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 173/340] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:51.756223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cdb25-8644-44f9-85ea-4031fba418f7 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:51.756423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2600e6f-8041-45cd-9ba7-a7f8b2011130 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:51.756817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cdb25-8644-44f9-85ea-4031fba418f7 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:51.756817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91cdb25-8644-44f9-85ea-4031fba418f7 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:51.761895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 174/341] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:51.765862 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4065a-9e93-402a-ad8e-5518fb8f4871 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:51.766372 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4065a-9e93-402a-ad8e-5518fb8f4871 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:51.766821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4065a-9e93-402a-ad8e-5518fb8f4871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:51.796469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-95a4065a-9e93-402a-ad8e-5518fb8f4871 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:51.797999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-95a4065a-9e93-402a-ad8e-5518fb8f4871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:51.814211 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9111ea-bfad-4215-bb77-0c55f6998b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:51.814947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9111ea-bfad-4215-bb77-0c55f6998b8e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:51.815379 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9111ea-bfad-4215-bb77-0c55f6998b8e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:51.815759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9111ea-bfad-4215-bb77-0c55f6998b8e None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:51.816506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9111ea-bfad-4215-bb77-0c55f6998b8e None admin] RBAC: Adding query filter params (name=member) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:51.816884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9111ea-bfad-4215-bb77-0c55f6998b8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:51.818948 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9111ea-bfad-4215-bb77-0c55f6998b8e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:51.828341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 168/342] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:51.829731 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e915e-16c8-4908-9d5b-a9bda62a6cfd None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:51.832918 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e915e-16c8-4908-9d5b-a9bda62a6cfd None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:51.833307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e915e-16c8-4908-9d5b-a9bda62a6cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:51.878739 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-120e915e-16c8-4908-9d5b-a9bda62a6cfd None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:51.879991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-120e915e-16c8-4908-9d5b-a9bda62a6cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:51.926347 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 175/343] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:51.927308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4065a-9e93-402a-ad8e-5518fb8f4871 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:52.020596 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 169/344] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:52.119897 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4065a-9e93-402a-ad8e-5518fb8f4871 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:52.128391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:52.166483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ea220-6852-43f1-ae7f-234ea9cc0711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:52.167100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ea220-6852-43f1-ae7f-234ea9cc0711 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:52.167376 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ea220-6852-43f1-ae7f-234ea9cc0711 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:52.167646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ea220-6852-43f1-ae7f-234ea9cc0711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:52.245535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ea220-6852-43f1-ae7f-234ea9cc0711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:52.247021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61ea220-6852-43f1-ae7f-234ea9cc0711 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:52.317448 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e915e-16c8-4908-9d5b-a9bda62a6cfd None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:52.351723 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:52.360033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 176/345] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:52.360960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ea220-6852-43f1-ae7f-234ea9cc0711 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:52.390170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:52.481944 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ea220-6852-43f1-ae7f-234ea9cc0711 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:52.511544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e915e-16c8-4908-9d5b-a9bda62a6cfd None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:52.523893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee96a27-cc3f-4863-b447-14f4aea1d26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:52.524543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee96a27-cc3f-4863-b447-14f4aea1d26b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:52.524820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee96a27-cc3f-4863-b447-14f4aea1d26b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:52.525090 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee96a27-cc3f-4863-b447-14f4aea1d26b None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:52.541563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee96a27-cc3f-4863-b447-14f4aea1d26b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-7ee96a27-cc3f-4863-b447-14f4aea1d26b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:52.548572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:52.549663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:52.550724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:52.550724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:52.550724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:52.550724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:52.550724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:52.550724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:52.550724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:52.550724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:52.550724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:52.550724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:52.550724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:52.550724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:52.552260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b32485-4087-4655-8b5a-b586563889ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:52.552903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b32485-4087-4655-8b5a-b586563889ff None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:52.553172 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b32485-4087-4655-8b5a-b586563889ff None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:52.553515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b32485-4087-4655-8b5a-b586563889ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:52.553662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 177/346] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:52.558295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee96a27-cc3f-4863-b447-14f4aea1d26b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:52.568699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b32485-4087-4655-8b5a-b586563889ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-f4b32485-4087-4655-8b5a-b586563889ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:52.573873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:52.575055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:52.576141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:52.576141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:52.576141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:52.576141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:52.576141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:52.576141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:52.576141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:52.576141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:52.576141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:52.576141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:52.576141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 19:33:52.576141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:52.576141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 170/347] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:33:52.576141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b32485-4087-4655-8b5a-b586563889ff None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:52.639441 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:52.681399 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:52.692821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee96a27-cc3f-4863-b447-14f4aea1d26b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:52.707214 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b32485-4087-4655-8b5a-b586563889ff None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:52.740479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de4999-2e5a-4f85-9e6f-91b60ca08076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:52.741660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de4999-2e5a-4f85-9e6f-91b60ca08076 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:52.742492 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de4999-2e5a-4f85-9e6f-91b60ca08076 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:52.743195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de4999-2e5a-4f85-9e6f-91b60ca08076 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:52.745200 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de4999-2e5a-4f85-9e6f-91b60ca08076 None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:52.747160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de4999-2e5a-4f85-9e6f-91b60ca08076 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:52.749683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 178/348] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:52 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:52.750622 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de4999-2e5a-4f85-9e6f-91b60ca08076 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:52.755554 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6bfa2-9824-4fd5-a232-ff8ed3e4adb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:52.756389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6bfa2-9824-4fd5-a232-ff8ed3e4adb2 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:52.756815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6bfa2-9824-4fd5-a232-ff8ed3e4adb2 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:52.757886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6bfa2-9824-4fd5-a232-ff8ed3e4adb2 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:52.758739 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6bfa2-9824-4fd5-a232-ff8ed3e4adb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:52.759241 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6bfa2-9824-4fd5-a232-ff8ed3e4adb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:52.761829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6bfa2-9824-4fd5-a232-ff8ed3e4adb2 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:52.773684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 171/349] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:52.779324 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6bfa2-9824-4fd5-a232-ff8ed3e4adb2 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:52.888778 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de4999-2e5a-4f85-9e6f-91b60ca08076 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:52.932356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7346d-99ec-41e4-949e-30827a021a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:52.933142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7346d-99ec-41e4-949e-30827a021a8a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:52.933502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6bfa2-9824-4fd5-a232-ff8ed3e4adb2 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:52.933626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7346d-99ec-41e4-949e-30827a021a8a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:52.933990 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7346d-99ec-41e4-949e-30827a021a8a None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:52.934826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7346d-99ec-41e4-949e-30827a021a8a None admin] RBAC: Adding query filter params (name=reader) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:52.935229 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7346d-99ec-41e4-949e-30827a021a8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:52.939021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7346d-99ec-41e4-949e-30827a021a8a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:52.944235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:52.955479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 179/350] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:52.956568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7346d-99ec-41e4-949e-30827a021a8a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:52.984157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18236f6-731f-4b7c-a527-d5570a7c8af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:52.985094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18236f6-731f-4b7c-a527-d5570a7c8af0 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:52.985569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18236f6-731f-4b7c-a527-d5570a7c8af0 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:52.986117 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18236f6-731f-4b7c-a527-d5570a7c8af0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:52.994414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:53.000562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18236f6-731f-4b7c-a527-d5570a7c8af0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-c18236f6-731f-4b7c-a527-d5570a7c8af0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:53.006168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:53.010308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:53.011148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:53.011148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:53.011148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:53.011148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:53.011148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:53.011148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:53.011148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:53.011148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:53.011148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:53.011148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:53.011148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:53.011148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:53.011148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 172/351] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:33:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:53.012785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18236f6-731f-4b7c-a527-d5570a7c8af0 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:53.093409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7346d-99ec-41e4-949e-30827a021a8a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:53.146012 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18236f6-731f-4b7c-a527-d5570a7c8af0 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:53.147471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5197e96-5ab0-4bde-97e8-c3948d34615c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:53.148123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5197e96-5ab0-4bde-97e8-c3948d34615c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:53.148400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5197e96-5ab0-4bde-97e8-c3948d34615c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:53.148667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5197e96-5ab0-4bde-97e8-c3948d34615c None admin] PATH_INFO: `/v3/groups/0224dbb2925e4a2b99e29e07cefccd01` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:53.157375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5197e96-5ab0-4bde-97e8-c3948d34615c None admin] RBAC: Authorizing `identity:get_group(group_id=0224dbb2925e4a2b99e29e07cefccd01)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:53.158988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5197e96-5ab0-4bde-97e8-c3948d34615c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:53.160837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 180/352] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:33:52 2022] GET /identity/v3/groups/0224dbb2925e4a2b99e29e07cefccd01 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:53.161447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5197e96-5ab0-4bde-97e8-c3948d34615c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:53.195414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30397e6d-5013-4ace-be14-c157d09d2466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:53.196369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30397e6d-5013-4ace-be14-c157d09d2466 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:53.196878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30397e6d-5013-4ace-be14-c157d09d2466 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:53.197350 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30397e6d-5013-4ace-be14-c157d09d2466 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:53.239561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:53.292590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:53.295165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5197e96-5ab0-4bde-97e8-c3948d34615c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:53.297854 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30397e6d-5013-4ace-be14-c157d09d2466 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:53.299536 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30397e6d-5013-4ace-be14-c157d09d2466 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:53.334912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf137133-570a-424d-ac97-00f7d6f41711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:53.335569 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf137133-570a-424d-ac97-00f7d6f41711 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:53.335849 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf137133-570a-424d-ac97-00f7d6f41711 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:53.336122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf137133-570a-424d-ac97-00f7d6f41711 None admin] PATH_INFO: `/v3/users/0e90add7e5a248a9b6d8de822459a1f7` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:53.337313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf137133-570a-424d-ac97-00f7d6f41711 None admin] RBAC: Authorizing `identity:get_user(user_id=0e90add7e5a248a9b6d8de822459a1f7)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:53.338816 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf137133-570a-424d-ac97-00f7d6f41711 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:53.340638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 181/353] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:33:53 2022] GET /identity/v3/users/0e90add7e5a248a9b6d8de822459a1f7 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:53.341277 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf137133-570a-424d-ac97-00f7d6f41711 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:53.434656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 173/354] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:53.436631 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9841542-75a0-4270-89d6-b2f1a16391d7 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:53.436916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9841542-75a0-4270-89d6-b2f1a16391d7 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:53.437188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9841542-75a0-4270-89d6-b2f1a16391d7 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:53.443102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 174/355] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:53.443703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9841542-75a0-4270-89d6-b2f1a16391d7 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:53.460267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf137133-570a-424d-ac97-00f7d6f41711 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:53.491704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:53.502373 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711b6cf-7880-487c-8fd1-51577c53c074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:53.502983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711b6cf-7880-487c-8fd1-51577c53c074 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:53.503262 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711b6cf-7880-487c-8fd1-51577c53c074 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:53.503532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711b6cf-7880-487c-8fd1-51577c53c074 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:53.504192 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711b6cf-7880-487c-8fd1-51577c53c074 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:53.504492 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711b6cf-7880-487c-8fd1-51577c53c074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:53.506438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711b6cf-7880-487c-8fd1-51577c53c074 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:53.516167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 182/356] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:33:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:53.516995 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711b6cf-7880-487c-8fd1-51577c53c074 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:53.558552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:53.564209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9841542-75a0-4270-89d6-b2f1a16391d7 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:53.602585 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6f24c-437a-4b3b-81ca-d41865fd0468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:53.603515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6f24c-437a-4b3b-81ca-d41865fd0468 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:53.603798 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6f24c-437a-4b3b-81ca-d41865fd0468 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:53.604062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6f24c-437a-4b3b-81ca-d41865fd0468 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:53.604789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6f24c-437a-4b3b-81ca-d41865fd0468 None admin] RBAC: Adding query filter params (group.id=0224dbb2925e4a2b99e29e07cefccd01, role.id=efa4ca02b2ff4ad6b14d501d4d79bbea, scope.project.id=07768f903a224486a253fc7acc9c3360) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:53.605627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6f24c-437a-4b3b-81ca-d41865fd0468 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:53.607195 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6f24c-437a-4b3b-81ca-d41865fd0468 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:53.614925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 175/357] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 19:33:53 2022] GET /identity/v3/role_assignments?group.id=0224dbb2925e4a2b99e29e07cefccd01&scope.project.id=07768f903a224486a253fc7acc9c3360&role.id=efa4ca02b2ff4ad6b14d501d4d79bbea => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:53.614925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6f24c-437a-4b3b-81ca-d41865fd0468 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:53.648453 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711b6cf-7880-487c-8fd1-51577c53c074 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:53.702831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa46ef9-b51f-44a6-b606-b2403f7ee652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:53.703381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa46ef9-b51f-44a6-b606-b2403f7ee652 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:53.703660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa46ef9-b51f-44a6-b606-b2403f7ee652 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:53.703934 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa46ef9-b51f-44a6-b606-b2403f7ee652 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:53.704676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa46ef9-b51f-44a6-b606-b2403f7ee652 None admin] RBAC: Adding query filter params (role.id=35aa74738f2d4844af99831724f20d39, user.id=0e90add7e5a248a9b6d8de822459a1f7, scope.system=all) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:53.705007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa46ef9-b51f-44a6-b606-b2403f7ee652 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:53.706551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa46ef9-b51f-44a6-b606-b2403f7ee652 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:53.725493 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 183/358] 10.222.0.42 () {56 vars in 1308 bytes} [Wed Sep 14 19:33:53 2022] GET /identity/v3/role_assignments?user.id=0e90add7e5a248a9b6d8de822459a1f7&scope.system=all&role.id=35aa74738f2d4844af99831724f20d39 => generated 510 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:53.726821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746a59d-73bf-4597-9e38-f009edda4743 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:53.727098 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746a59d-73bf-4597-9e38-f009edda4743 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:53.727378 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e746a59d-73bf-4597-9e38-f009edda4743 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:53.728932 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 184/359] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:53.730076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292d787-97e6-4a7d-9c1a-de8e55a081a2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:53.730358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292d787-97e6-4a7d-9c1a-de8e55a081a2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:53.730630 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292d787-97e6-4a7d-9c1a-de8e55a081a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:53.751913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6f24c-437a-4b3b-81ca-d41865fd0468 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:53.754536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c292d787-97e6-4a7d-9c1a-de8e55a081a2 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:53.755786 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c292d787-97e6-4a7d-9c1a-de8e55a081a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:53.805231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a32e33-f2ad-458d-ac66-8554f0271cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:53.805970 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a32e33-f2ad-458d-ac66-8554f0271cd9 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:53.806218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a32e33-f2ad-458d-ac66-8554f0271cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:53.806493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a32e33-f2ad-458d-ac66-8554f0271cd9 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:53.807687 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a32e33-f2ad-458d-ac66-8554f0271cd9 None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:53.809223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a32e33-f2ad-458d-ac66-8554f0271cd9 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:53.812694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 176/360] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:53 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:53.813567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a32e33-f2ad-458d-ac66-8554f0271cd9 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:53.819091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:53.851099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 185/361] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:53.852877 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866bb13-86a9-4f81-bb38-c2dd41e8f561 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:53.853459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866bb13-86a9-4f81-bb38-c2dd41e8f561 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:53.855110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866bb13-86a9-4f81-bb38-c2dd41e8f561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:53.879334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:53.883933 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a866bb13-86a9-4f81-bb38-c2dd41e8f561 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:53.885455 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a866bb13-86a9-4f81-bb38-c2dd41e8f561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:53.973041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 186/362] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:53.974317 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866bb13-86a9-4f81-bb38-c2dd41e8f561 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:53.980363 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a32e33-f2ad-458d-ac66-8554f0271cd9 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:54.030489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db32ce-4594-4ea4-a31d-3d724fab6e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:54.031106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db32ce-4594-4ea4-a31d-3d724fab6e6b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:54.031481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db32ce-4594-4ea4-a31d-3d724fab6e6b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:54.031762 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db32ce-4594-4ea4-a31d-3d724fab6e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:54.114088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db32ce-4594-4ea4-a31d-3d724fab6e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:54.116293 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db32ce-4594-4ea4-a31d-3d724fab6e6b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:54.143835 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866bb13-86a9-4f81-bb38-c2dd41e8f561 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:54.151415 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:54.160295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:54.207549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbf91c-345a-497c-893d-150750fd9350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:54.208620 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbf91c-345a-497c-893d-150750fd9350 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:54.209074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbf91c-345a-497c-893d-150750fd9350 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:54.209514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbf91c-345a-497c-893d-150750fd9350 None admin] PATH_INFO: `/v3/groups/0224dbb2925e4a2b99e29e07cefccd01` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:54.213130 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbf91c-345a-497c-893d-150750fd9350 None admin] RBAC: Authorizing `identity:get_group(group_id=0224dbb2925e4a2b99e29e07cefccd01)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:54.214889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fbf91c-345a-497c-893d-150750fd9350 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:54.217412 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 187/363] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:33:53 2022] GET /identity/v3/groups/0224dbb2925e4a2b99e29e07cefccd01 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:54.223360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbf91c-345a-497c-893d-150750fd9350 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:54.238090 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 177/364] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:54.244703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db32ce-4594-4ea4-a31d-3d724fab6e6b None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:54.365527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbf91c-345a-497c-893d-150750fd9350 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:54.387249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db32ce-4594-4ea4-a31d-3d724fab6e6b None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:54.407921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c7597-9518-4b66-92dc-cfa71ab53c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:54.408529 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c7597-9518-4b66-92dc-cfa71ab53c5f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:54.408801 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c7597-9518-4b66-92dc-cfa71ab53c5f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:54.409068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3c7597-9518-4b66-92dc-cfa71ab53c5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:54.409969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c7597-9518-4b66-92dc-cfa71ab53c5f None admin] RBAC: Adding query filter params (group.id=0224dbb2925e4a2b99e29e07cefccd01, role.id=efa4ca02b2ff4ad6b14d501d4d79bbea, scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:54.410569 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c7597-9518-4b66-92dc-cfa71ab53c5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:54.411989 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3c7597-9518-4b66-92dc-cfa71ab53c5f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:54.418675 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 188/365] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 19:33:54 2022] GET /identity/v3/role_assignments?group.id=0224dbb2925e4a2b99e29e07cefccd01&scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552&role.id=efa4ca02b2ff4ad6b14d501d4d79bbea => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:54.419509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c7597-9518-4b66-92dc-cfa71ab53c5f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:54.432621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:54.448209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d23aa-6f8b-4d02-b7f8-374333c8295f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:54.448902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d23aa-6f8b-4d02-b7f8-374333c8295f None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:54.449176 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d23aa-6f8b-4d02-b7f8-374333c8295f None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:54.449442 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d23aa-6f8b-4d02-b7f8-374333c8295f None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:54.450690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d23aa-6f8b-4d02-b7f8-374333c8295f None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:54.452262 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d23aa-6f8b-4d02-b7f8-374333c8295f None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:54.454370 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 178/366] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:54 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:54.455220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d23aa-6f8b-4d02-b7f8-374333c8295f None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:54.483765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:54.562196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3c7597-9518-4b66-92dc-cfa71ab53c5f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:54.585185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d23aa-6f8b-4d02-b7f8-374333c8295f None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:54.602900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20546c-8888-488e-b91c-9336945451e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:54.604109 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20546c-8888-488e-b91c-9336945451e9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:54.604393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20546c-8888-488e-b91c-9336945451e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:54.604666 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20546c-8888-488e-b91c-9336945451e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:54.619035 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20546c-8888-488e-b91c-9336945451e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-df20546c-8888-488e-b91c-9336945451e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:54.625550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:54.626521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:54.627348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:54.627348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:54.627348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:54.627348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:54.627348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:54.627348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:54.627348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:54.627348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:54.627348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:54.627348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:54.627348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:54.627348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:33:54.627348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 189/367] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:33:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:54.628024 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20546c-8888-488e-b91c-9336945451e9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:54.631732 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2b4cf-38f7-4a3b-983e-c6b188b258cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:54.632633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2b4cf-38f7-4a3b-983e-c6b188b258cf None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:54.633111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2b4cf-38f7-4a3b-983e-c6b188b258cf None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:54.633638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2b4cf-38f7-4a3b-983e-c6b188b258cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:54.646458 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e2b4cf-38f7-4a3b-983e-c6b188b258cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-c9e2b4cf-38f7-4a3b-983e-c6b188b258cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:54.652938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:54.653977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:54.654916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:54.654916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:54.654916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:54.654916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:54.654916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:54.654916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:54.654916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:54.654916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:54.654916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:54.654916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:54.654916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:54.654916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:54.655850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 179/368] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:33:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:54.661755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2b4cf-38f7-4a3b-983e-c6b188b258cf None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:54.737850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:54.767126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:54.772317 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20546c-8888-488e-b91c-9336945451e9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:54.791457 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e2b4cf-38f7-4a3b-983e-c6b188b258cf None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:54.823925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a2247-6af6-40bc-84da-55ec02ff7700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:54.824677 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a2247-6af6-40bc-84da-55ec02ff7700 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:54.824953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a2247-6af6-40bc-84da-55ec02ff7700 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:54.825226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a2247-6af6-40bc-84da-55ec02ff7700 None admin] PATH_INFO: `/v3/groups/0224dbb2925e4a2b99e29e07cefccd01` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:54.826972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a2247-6af6-40bc-84da-55ec02ff7700 None admin] RBAC: Authorizing `identity:get_group(group_id=0224dbb2925e4a2b99e29e07cefccd01)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:54.828718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a2247-6af6-40bc-84da-55ec02ff7700 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:54.830614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 190/369] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:33:54 2022] GET /identity/v3/groups/0224dbb2925e4a2b99e29e07cefccd01 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:54.831490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a2247-6af6-40bc-84da-55ec02ff7700 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:54.833937 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b14105-f9a2-41ee-90ae-a3376333b2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:54.835244 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b14105-f9a2-41ee-90ae-a3376333b2fe None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:54.836543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b14105-f9a2-41ee-90ae-a3376333b2fe None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:54.836971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b14105-f9a2-41ee-90ae-a3376333b2fe None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:54.837869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b14105-f9a2-41ee-90ae-a3376333b2fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:54.838351 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b14105-f9a2-41ee-90ae-a3376333b2fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:54.839854 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b14105-f9a2-41ee-90ae-a3376333b2fe None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:54.851465 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 180/370] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:33:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:54.852685 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b14105-f9a2-41ee-90ae-a3376333b2fe None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:54.953363 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a2247-6af6-40bc-84da-55ec02ff7700 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:54.981857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b14105-f9a2-41ee-90ae-a3376333b2fe None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:55.001409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea4841-cf5a-4d33-aee9-4d283d87490a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:55.003568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea4841-cf5a-4d33-aee9-4d283d87490a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:55.004012 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea4841-cf5a-4d33-aee9-4d283d87490a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:55.004422 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea4841-cf5a-4d33-aee9-4d283d87490a None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:55.005216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea4841-cf5a-4d33-aee9-4d283d87490a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:55.005665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea4841-cf5a-4d33-aee9-4d283d87490a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:55.006268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:55.007829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea4841-cf5a-4d33-aee9-4d283d87490a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:55.019568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 191/371] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:55.020597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea4841-cf5a-4d33-aee9-4d283d87490a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:55.024826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaf681-c141-4daf-b523-82b616500919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:55.025600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaf681-c141-4daf-b523-82b616500919 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:55.026060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaf681-c141-4daf-b523-82b616500919 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:55.026789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaf681-c141-4daf-b523-82b616500919 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:55.028674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deaf681-c141-4daf-b523-82b616500919 None admin] RBAC: Adding query filter params (group.id=0224dbb2925e4a2b99e29e07cefccd01, role.id=36b107b4892f47f8af6b65e1d1d94bb0, scope.project.id=07768f903a224486a253fc7acc9c3360) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:55.029649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deaf681-c141-4daf-b523-82b616500919 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:55.031264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deaf681-c141-4daf-b523-82b616500919 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:55.040579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 181/372] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 19:33:54 2022] GET /identity/v3/role_assignments?group.id=0224dbb2925e4a2b99e29e07cefccd01&scope.project.id=07768f903a224486a253fc7acc9c3360&role.id=36b107b4892f47f8af6b65e1d1d94bb0 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:55.041898 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaf681-c141-4daf-b523-82b616500919 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:55.060021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:55.151325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea4841-cf5a-4d33-aee9-4d283d87490a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:55.179641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaf681-c141-4daf-b523-82b616500919 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:55.197176 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a514b-0745-44ab-9d9c-93eff2ff8426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:55.197911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a514b-0745-44ab-9d9c-93eff2ff8426 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:55.198201 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a514b-0745-44ab-9d9c-93eff2ff8426 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:55.198468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a514b-0745-44ab-9d9c-93eff2ff8426 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:55.223998 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bdad4-32af-41e2-ae4e-2ae232cd6b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:55.225032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bdad4-32af-41e2-ae4e-2ae232cd6b3e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:55.225503 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bdad4-32af-41e2-ae4e-2ae232cd6b3e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:55.225999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748bdad4-32af-41e2-ae4e-2ae232cd6b3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:55.287317 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a514b-0745-44ab-9d9c-93eff2ff8426 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:55.288870 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a514b-0745-44ab-9d9c-93eff2ff8426 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:55.300813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:55.320677 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748bdad4-32af-41e2-ae4e-2ae232cd6b3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:55.321857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748bdad4-32af-41e2-ae4e-2ae232cd6b3e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:55.342213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:55.437862 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 192/373] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:55.443672 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a514b-0745-44ab-9d9c-93eff2ff8426 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:55.457017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 182/374] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:33:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:33:55.467005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bdad4-32af-41e2-ae4e-2ae232cd6b3e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:55.553644 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a514b-0745-44ab-9d9c-93eff2ff8426 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:55.583861 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:55.589811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9baf9-4f23-4025-91b0-da3fd9100b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:55.590645 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9baf9-4f23-4025-91b0-da3fd9100b86 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:55.590998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9baf9-4f23-4025-91b0-da3fd9100b86 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:55.591348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9baf9-4f23-4025-91b0-da3fd9100b86 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:55.592842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9baf9-4f23-4025-91b0-da3fd9100b86 None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:55.594414 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9baf9-4f23-4025-91b0-da3fd9100b86 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:55.594817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:55.596480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 193/375] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:55 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:55.602188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9baf9-4f23-4025-91b0-da3fd9100b86 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:55.616972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748bdad4-32af-41e2-ae4e-2ae232cd6b3e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:55.671455 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c52b82-e3ef-43d3-af58-c87180bd0318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:55.672101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c52b82-e3ef-43d3-af58-c87180bd0318 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:55.672383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c52b82-e3ef-43d3-af58-c87180bd0318 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:55.672659 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c52b82-e3ef-43d3-af58-c87180bd0318 None admin] PATH_INFO: `/v3/projects/2a8f053b53ae4aff97a668669e77d899` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:55.674886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c52b82-e3ef-43d3-af58-c87180bd0318 None admin] RBAC: Authorizing `identity:get_project(project_id=2a8f053b53ae4aff97a668669e77d899)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:55.676476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c52b82-e3ef-43d3-af58-c87180bd0318 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:55.678468 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 183/376] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:55 2022] GET /identity/v3/projects/2a8f053b53ae4aff97a668669e77d899 => generated 344 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:55.684032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c52b82-e3ef-43d3-af58-c87180bd0318 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:55.711871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9baf9-4f23-4025-91b0-da3fd9100b86 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:55.747733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc783f72-fd0a-4b5d-9319-cc43f80d8c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:55.748937 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc783f72-fd0a-4b5d-9319-cc43f80d8c71 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:55.749216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc783f72-fd0a-4b5d-9319-cc43f80d8c71 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:55.749486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc783f72-fd0a-4b5d-9319-cc43f80d8c71 None admin] PATH_INFO: `/v3/groups/0224dbb2925e4a2b99e29e07cefccd01` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:55.750654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc783f72-fd0a-4b5d-9319-cc43f80d8c71 None admin] RBAC: Authorizing `identity:get_group(group_id=0224dbb2925e4a2b99e29e07cefccd01)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:55.752113 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc783f72-fd0a-4b5d-9319-cc43f80d8c71 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:55.754008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 194/377] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:33:55 2022] GET /identity/v3/groups/0224dbb2925e4a2b99e29e07cefccd01 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:55.760001 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc783f72-fd0a-4b5d-9319-cc43f80d8c71 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:55.839432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:55.887539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c52b82-e3ef-43d3-af58-c87180bd0318 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:55.912763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc783f72-fd0a-4b5d-9319-cc43f80d8c71 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:55.955842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e12f1f-5c9f-4188-8198-566af80e6d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:55.956457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e12f1f-5c9f-4188-8198-566af80e6d59 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:55.956736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e12f1f-5c9f-4188-8198-566af80e6d59 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:55.957009 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e12f1f-5c9f-4188-8198-566af80e6d59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:55.957725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e12f1f-5c9f-4188-8198-566af80e6d59 None admin] RBAC: Adding query filter params (group.id=0224dbb2925e4a2b99e29e07cefccd01, role.id=36b107b4892f47f8af6b65e1d1d94bb0, scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:55.958059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e12f1f-5c9f-4188-8198-566af80e6d59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:55.960890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e12f1f-5c9f-4188-8198-566af80e6d59 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:55.963208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072faf0-9d53-4a44-8d72-ef3eabaffe99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:55.963845 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072faf0-9d53-4a44-8d72-ef3eabaffe99 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:55.964122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072faf0-9d53-4a44-8d72-ef3eabaffe99 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:55.964393 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072faf0-9d53-4a44-8d72-ef3eabaffe99 None admin] PATH_INFO: `/v3/groups/36c71cc3f53742a99696232600eae5eb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:55.973463 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072faf0-9d53-4a44-8d72-ef3eabaffe99 None admin] RBAC: Authorizing `identity:get_group(group_id=36c71cc3f53742a99696232600eae5eb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:55.975118 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072faf0-9d53-4a44-8d72-ef3eabaffe99 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:55.975240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 195/378] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 19:33:55 2022] GET /identity/v3/role_assignments?group.id=0224dbb2925e4a2b99e29e07cefccd01&scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552&role.id=36b107b4892f47f8af6b65e1d1d94bb0 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:55.976826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 184/379] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:33:55 2022] GET /identity/v3/groups/36c71cc3f53742a99696232600eae5eb => generated 229 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:55.986097 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e12f1f-5c9f-4188-8198-566af80e6d59 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:56.144188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e12f1f-5c9f-4188-8198-566af80e6d59 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:56.188537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:56.195141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6cb07a-dffd-45fb-a7a1-5d253b62dcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:56.195770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb07a-dffd-45fb-a7a1-5d253b62dcf4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:56.196051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb07a-dffd-45fb-a7a1-5d253b62dcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:56.196322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6cb07a-dffd-45fb-a7a1-5d253b62dcf4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:56.197059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cb07a-dffd-45fb-a7a1-5d253b62dcf4 None admin] RBAC: Adding query filter params (group.id=36c71cc3f53742a99696232600eae5eb, role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=2a8f053b53ae4aff97a668669e77d899) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:56.197369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cb07a-dffd-45fb-a7a1-5d253b62dcf4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:56.198847 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6cb07a-dffd-45fb-a7a1-5d253b62dcf4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:56.206302 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 196/380] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 19:33:55 2022] GET /identity/v3/role_assignments?group.id=36c71cc3f53742a99696232600eae5eb&scope.project.id=2a8f053b53ae4aff97a668669e77d899&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 260 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:56.298571 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78045525-e332-4094-855c-0d7269adc85c None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:56.298903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78045525-e332-4094-855c-0d7269adc85c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:56.299121 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78045525-e332-4094-855c-0d7269adc85c None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:56.300828 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 185/381] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:56.307920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88e86e-2330-40b2-9639-21099141a6f9 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:56.308225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88e86e-2330-40b2-9639-21099141a6f9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:56.308487 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c88e86e-2330-40b2-9639-21099141a6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:56.332360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6c88e86e-2330-40b2-9639-21099141a6f9 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:56.333678 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6c88e86e-2330-40b2-9639-21099141a6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:56.427511 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 197/382] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:56.753403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78045525-e332-4094-855c-0d7269adc85c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:56.775027 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:56.889209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78045525-e332-4094-855c-0d7269adc85c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:56.932736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e57e10-cf04-4aab-9915-a0925d1355e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:56.933360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e57e10-cf04-4aab-9915-a0925d1355e0 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:56.933634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e57e10-cf04-4aab-9915-a0925d1355e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:56.933925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e57e10-cf04-4aab-9915-a0925d1355e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:56.947350 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e57e10-cf04-4aab-9915-a0925d1355e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-a3e57e10-cf04-4aab-9915-a0925d1355e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:56.953088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:56.953989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:56.954809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:56.954809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:56.954809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:56.954809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:56.954809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:56.954809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:56.954809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:56.954809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:56.954809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:56.954809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:56.954809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:56.954809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:56.954809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 186/383] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:56.960316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88e86e-2330-40b2-9639-21099141a6f9 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:57.076866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c88e86e-2330-40b2-9639-21099141a6f9 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:57.105336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:57.120972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2a8f-3c23-47fe-a8ca-5afa00857cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:57.121604 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2a8f-3c23-47fe-a8ca-5afa00857cf0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:57.127676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2a8f-3c23-47fe-a8ca-5afa00857cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:57.127676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e2a8f-3c23-47fe-a8ca-5afa00857cf0 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:57.127676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e2a8f-3c23-47fe-a8ca-5afa00857cf0 None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:57.127676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e2a8f-3c23-47fe-a8ca-5afa00857cf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:57.127676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e2a8f-3c23-47fe-a8ca-5afa00857cf0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:57.137890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 198/384] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:57.139728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e57e10-cf04-4aab-9915-a0925d1355e0 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:57.290871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:57.321403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e57e10-cf04-4aab-9915-a0925d1355e0 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:57.380528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7f9bc-349e-4756-a662-eb3488d40c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:57.381081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7f9bc-349e-4756-a662-eb3488d40c4f None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:57.381356 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7f9bc-349e-4756-a662-eb3488d40c4f None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:57.381626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7f9bc-349e-4756-a662-eb3488d40c4f None admin] PATH_INFO: `/v3/groups/0224dbb2925e4a2b99e29e07cefccd01` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:57.382758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7f9bc-349e-4756-a662-eb3488d40c4f None admin] RBAC: Authorizing `identity:get_group(group_id=0224dbb2925e4a2b99e29e07cefccd01)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:57.384190 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7f9bc-349e-4756-a662-eb3488d40c4f None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:57.385752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 187/385] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:33:57 2022] GET /identity/v3/groups/0224dbb2925e4a2b99e29e07cefccd01 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:57.391429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2a8f-3c23-47fe-a8ca-5afa00857cf0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:57.519342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e2a8f-3c23-47fe-a8ca-5afa00857cf0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:57.568307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9df3cc-5d45-4e4c-9995-c6d088bdfc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:57.568966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9df3cc-5d45-4e4c-9995-c6d088bdfc70 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:57.569247 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9df3cc-5d45-4e4c-9995-c6d088bdfc70 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:57.569525 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9df3cc-5d45-4e4c-9995-c6d088bdfc70 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:57.574586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9df3cc-5d45-4e4c-9995-c6d088bdfc70 None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:57.576130 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9df3cc-5d45-4e4c-9995-c6d088bdfc70 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:57.577843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 199/386] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:57 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:57.583781 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7f9bc-349e-4756-a662-eb3488d40c4f None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:57.710044 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdab74a-f76f-4868-8341-23fc6c4e6248 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:57.710384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdab74a-f76f-4868-8341-23fc6c4e6248 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:57.710558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdab74a-f76f-4868-8341-23fc6c4e6248 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:57.713104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 200/387] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:57.720803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca028d38-5234-49cb-aab9-d1b64da90654 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:57.724412 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca028d38-5234-49cb-aab9-d1b64da90654 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:57.724911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca028d38-5234-49cb-aab9-d1b64da90654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:57.751502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-ca028d38-5234-49cb-aab9-d1b64da90654 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:57.753148 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ca028d38-5234-49cb-aab9-d1b64da90654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:57.769773 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7f9bc-349e-4756-a662-eb3488d40c4f None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:57.834216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:57.835281 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7c84d-be53-4483-bbe8-a1b0ca5ff148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:57.839353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7c84d-be53-4483-bbe8-a1b0ca5ff148 None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:57.839934 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7c84d-be53-4483-bbe8-a1b0ca5ff148 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:57.840459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7c84d-be53-4483-bbe8-a1b0ca5ff148 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360/groups/0224dbb2925e4a2b99e29e07cefccd01/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:57.844226 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7c84d-be53-4483-bbe8-a1b0ca5ff148 None admin] RBAC: Authorizing `identity:create_grant(project_id=07768f903a224486a253fc7acc9c3360, group_id=0224dbb2925e4a2b99e29e07cefccd01, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:57.847496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7c84d-be53-4483-bbe8-a1b0ca5ff148 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:57.865992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 201/388] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:57.946043 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5a2c7-4649-41d8-b11b-8921f1b741f0 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:57.946637 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5a2c7-4649-41d8-b11b-8921f1b741f0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:57.947203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb5a2c7-4649-41d8-b11b-8921f1b741f0 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:57.948974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 202/389] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:33:57.953504 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80a31a-6c83-45e1-a0d3-f482a9044fcf None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:57.954043 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80a31a-6c83-45e1-a0d3-f482a9044fcf None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:57.954538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80a31a-6c83-45e1-a0d3-f482a9044fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:57.960311 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 188/390] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Sep 14 19:33:57 2022] PUT /identity/v3/projects/07768f903a224486a253fc7acc9c3360/groups/0224dbb2925e4a2b99e29e07cefccd01/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:57.978267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4f80a31a-6c83-45e1-a0d3-f482a9044fcf None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:57.980309 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4f80a31a-6c83-45e1-a0d3-f482a9044fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:58.068755 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 203/391] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:33:58.195837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7c84d-be53-4483-bbe8-a1b0ca5ff148 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:58.199743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:58.313962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7c84d-be53-4483-bbe8-a1b0ca5ff148 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:58.349682 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e91be-a755-4401-aaf7-6c27af867551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:58.350423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e91be-a755-4401-aaf7-6c27af867551 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:58.350703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e91be-a755-4401-aaf7-6c27af867551 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:58.351037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4e91be-a755-4401-aaf7-6c27af867551 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:58.364868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4e91be-a755-4401-aaf7-6c27af867551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-2f4e91be-a755-4401-aaf7-6c27af867551 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:58.370366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:58.371276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:58.372057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:58.372057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:58.372057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:58.372057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:58.372057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:58.372057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:58.372057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:58.372057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:58.372057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:58.372057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:58.372057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:33:58.372057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:58.375768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 189/392] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:58.377765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80a31a-6c83-45e1-a0d3-f482a9044fcf None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:58.466562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e91be-a755-4401-aaf7-6c27af867551 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:58.523269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80a31a-6c83-45e1-a0d3-f482a9044fcf None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:58.554172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:58.570486 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:58.587479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef95c0-4c37-4938-8209-2a583a075aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:58.589125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef95c0-4c37-4938-8209-2a583a075aa5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:58.589499 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef95c0-4c37-4938-8209-2a583a075aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:58.589784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef95c0-4c37-4938-8209-2a583a075aa5 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:58.590473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef95c0-4c37-4938-8209-2a583a075aa5 None admin] RBAC: Adding query filter params (name=member) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:58.590775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef95c0-4c37-4938-8209-2a583a075aa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:58.592241 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef95c0-4c37-4938-8209-2a583a075aa5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:58.610836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4e91be-a755-4401-aaf7-6c27af867551 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:58.612628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 204/393] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:33:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:58.630105 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef95c0-4c37-4938-8209-2a583a075aa5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:58.663401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e05a01-f66f-4967-9ac4-001bab0556f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:58.664039 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e05a01-f66f-4967-9ac4-001bab0556f0 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:58.664318 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e05a01-f66f-4967-9ac4-001bab0556f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:58.664590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e05a01-f66f-4967-9ac4-001bab0556f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:58.682362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e05a01-f66f-4967-9ac4-001bab0556f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-16e05a01-f66f-4967-9ac4-001bab0556f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:58.687759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:58.688663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:58.689467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:58.689467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:58.689467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:58.689467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:58.689467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:58.689467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:58.689467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:58.689467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:58.689467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:58.689467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:58.689467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:58.689467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:58.698150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 190/394] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:33:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:33:58.699020 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e05a01-f66f-4967-9ac4-001bab0556f0 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:58.812236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef95c0-4c37-4938-8209-2a583a075aa5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:58.840831 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e05a01-f66f-4967-9ac4-001bab0556f0 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:58.872275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:58.885330 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa6362-bd73-47f2-b1c4-48ad4ebc7bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:58.886042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa6362-bd73-47f2-b1c4-48ad4ebc7bef None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:58.886323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa6362-bd73-47f2-b1c4-48ad4ebc7bef None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:58.887866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa6362-bd73-47f2-b1c4-48ad4ebc7bef None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:58.888118 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa6362-bd73-47f2-b1c4-48ad4ebc7bef None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:58.888479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa6362-bd73-47f2-b1c4-48ad4ebc7bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:58.890154 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa6362-bd73-47f2-b1c4-48ad4ebc7bef None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:58.899467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 191/395] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:33:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:58.907412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa6362-bd73-47f2-b1c4-48ad4ebc7bef None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:58.910607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1b62a-8b77-41a9-bd4a-4b4f24692365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:58.911364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1b62a-8b77-41a9-bd4a-4b4f24692365 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:58.911644 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1b62a-8b77-41a9-bd4a-4b4f24692365 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:58.911914 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1b62a-8b77-41a9-bd4a-4b4f24692365 None admin] PATH_INFO: `/v3/groups/0224dbb2925e4a2b99e29e07cefccd01` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:58.917192 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1b62a-8b77-41a9-bd4a-4b4f24692365 None admin] RBAC: Authorizing `identity:get_group(group_id=0224dbb2925e4a2b99e29e07cefccd01)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:58.918743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba1b62a-8b77-41a9-bd4a-4b4f24692365 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:58.920836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 205/396] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:33:58 2022] GET /identity/v3/groups/0224dbb2925e4a2b99e29e07cefccd01 => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:58.922063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec211e-669c-4cf8-b410-37844ba2bd47 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:58.922340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec211e-669c-4cf8-b410-37844ba2bd47 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:58.922608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec211e-669c-4cf8-b410-37844ba2bd47 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:58.924185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 206/397] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:58.929436 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec211e-669c-4cf8-b410-37844ba2bd47 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:58.993861 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:59.020166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa6362-bd73-47f2-b1c4-48ad4ebc7bef None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:59.062145 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bb6a1a-66be-43d2-ab72-3688011001b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:59.063340 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb6a1a-66be-43d2-ab72-3688011001b1 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:59.063776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb6a1a-66be-43d2-ab72-3688011001b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:59.064445 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bb6a1a-66be-43d2-ab72-3688011001b1 None admin] PATH_INFO: `/v3/groups/0224dbb2925e4a2b99e29e07cefccd01` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:59.066495 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb6a1a-66be-43d2-ab72-3688011001b1 None admin] RBAC: Authorizing `identity:get_group(group_id=0224dbb2925e4a2b99e29e07cefccd01)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:59.068691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bb6a1a-66be-43d2-ab72-3688011001b1 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:59.069182 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec211e-669c-4cf8-b410-37844ba2bd47 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:59.071383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 192/398] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:33:58 2022] GET /identity/v3/groups/0224dbb2925e4a2b99e29e07cefccd01 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:59.073681 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1213e3-c6d5-4ffe-9a7b-1a19f24e5f06 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:59.078195 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1213e3-c6d5-4ffe-9a7b-1a19f24e5f06 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:59.078195 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1213e3-c6d5-4ffe-9a7b-1a19f24e5f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:59.094782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-8d1213e3-c6d5-4ffe-9a7b-1a19f24e5f06 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:59.095911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8d1213e3-c6d5-4ffe-9a7b-1a19f24e5f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:59.106653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d26ca-f055-43c0-a73c-2431f2bba50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:59.108032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d26ca-f055-43c0-a73c-2431f2bba50b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:59.108313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d26ca-f055-43c0-a73c-2431f2bba50b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:59.108627 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77d26ca-f055-43c0-a73c-2431f2bba50b None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:59.110215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d26ca-f055-43c0-a73c-2431f2bba50b None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:59.112108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77d26ca-f055-43c0-a73c-2431f2bba50b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:59.118993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 207/399] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:58 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:59.118993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d26ca-f055-43c0-a73c-2431f2bba50b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:59.154834 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:59.173656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 193/400] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:59.174765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63a7a1-e5a1-432b-9f96-7465cfcaff2e None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:59.175048 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63a7a1-e5a1-432b-9f96-7465cfcaff2e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:59.175317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd63a7a1-e5a1-432b-9f96-7465cfcaff2e None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:59.176674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 194/401] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:33:59.181190 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63a7a1-e5a1-432b-9f96-7465cfcaff2e None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:59.241789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77d26ca-f055-43c0-a73c-2431f2bba50b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:59.256164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:59.277772 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2913a673-73f3-475b-a6ea-b6ad512d6e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:59.278486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913a673-73f3-475b-a6ea-b6ad512d6e75 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:59.278764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913a673-73f3-475b-a6ea-b6ad512d6e75 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:59.279071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2913a673-73f3-475b-a6ea-b6ad512d6e75 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:59.280244 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913a673-73f3-475b-a6ea-b6ad512d6e75 None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:59.281896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2913a673-73f3-475b-a6ea-b6ad512d6e75 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:59.283625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 208/402] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:33:59 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:33:59.284837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19ea4d-974a-423b-878c-90899c5c748c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:59.285118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19ea4d-974a-423b-878c-90899c5c748c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:59.285397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19ea4d-974a-423b-878c-90899c5c748c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:59.305568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4b19ea4d-974a-423b-878c-90899c5c748c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:33:59.306956 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4b19ea4d-974a-423b-878c-90899c5c748c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:33:59.339100 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd63a7a1-e5a1-432b-9f96-7465cfcaff2e None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:59.380233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1521ee-9fcf-44b6-b9e9-f7d2a7eb2df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:59.381487 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1521ee-9fcf-44b6-b9e9-f7d2a7eb2df5 None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:59.382249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1521ee-9fcf-44b6-b9e9-f7d2a7eb2df5 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:59.383155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1521ee-9fcf-44b6-b9e9-f7d2a7eb2df5 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552/groups/0224dbb2925e4a2b99e29e07cefccd01/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:59.386202 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1521ee-9fcf-44b6-b9e9-f7d2a7eb2df5 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5fbdcb3c4884d59ab798fc1c9752552, group_id=0224dbb2925e4a2b99e29e07cefccd01, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:59.389001 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1521ee-9fcf-44b6-b9e9-f7d2a7eb2df5 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:59.398134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 209/403] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:33:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:33:59.399306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19ea4d-974a-423b-878c-90899c5c748c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:59.463603 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 195/404] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Sep 14 19:33:59 2022] PUT /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552/groups/0224dbb2925e4a2b99e29e07cefccd01/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:59.464938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1521ee-9fcf-44b6-b9e9-f7d2a7eb2df5 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:59.526141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19ea4d-974a-423b-878c-90899c5c748c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:59.527855 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:59.541183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:59.591106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc733255-c19f-4d51-9c12-0ef8ce63f1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:59.591856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc733255-c19f-4d51-9c12-0ef8ce63f1a5 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:59.592134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc733255-c19f-4d51-9c12-0ef8ce63f1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:59.592431 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc733255-c19f-4d51-9c12-0ef8ce63f1a5 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360/groups/0224dbb2925e4a2b99e29e07cefccd01/roles/36b107b4892f47f8af6b65e1d1d94bb0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:59.598093 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc733255-c19f-4d51-9c12-0ef8ce63f1a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=07768f903a224486a253fc7acc9c3360, group_id=0224dbb2925e4a2b99e29e07cefccd01, role_id=36b107b4892f47f8af6b65e1d1d94bb0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:59.599556 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc733255-c19f-4d51-9c12-0ef8ce63f1a5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:59.626881 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1521ee-9fcf-44b6-b9e9-f7d2a7eb2df5 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:59.671950 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854e087-1e8a-45b3-829a-e5667a9288ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:59.673037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e087-1e8a-45b3-829a-e5667a9288ba None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:59.673519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e087-1e8a-45b3-829a-e5667a9288ba None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:59.674037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854e087-1e8a-45b3-829a-e5667a9288ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:59.689876 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854e087-1e8a-45b3-829a-e5667a9288ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-6854e087-1e8a-45b3-829a-e5667a9288ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:59.696345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:59.697253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:59.698695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:59.698695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:59.698695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:59.698695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:59.698695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:59.698695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:59.698695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:59.698695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:59.698695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:59.698695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:59.698695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:33:59.698695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:59.699327 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 210/405] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Sep 14 19:33:59 2022] PUT /identity/v3/projects/07768f903a224486a253fc7acc9c3360/groups/0224dbb2925e4a2b99e29e07cefccd01/roles/36b107b4892f47f8af6b65e1d1d94bb0 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:33:59.701393 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 196/406] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:33:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:33:59.706860 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc733255-c19f-4d51-9c12-0ef8ce63f1a5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:59.716191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854e087-1e8a-45b3-829a-e5667a9288ba None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:59.838599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc733255-c19f-4d51-9c12-0ef8ce63f1a5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:59.868197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854e087-1e8a-45b3-829a-e5667a9288ba None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:33:59.875789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:59.880873 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f729c-5398-4895-b4a8-45fe30c81885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:59.881745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f729c-5398-4895-b4a8-45fe30c81885 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:59.882226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f729c-5398-4895-b4a8-45fe30c81885 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:59.882670 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f729c-5398-4895-b4a8-45fe30c81885 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:59.883532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f729c-5398-4895-b4a8-45fe30c81885 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:33:59.884080 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f729c-5398-4895-b4a8-45fe30c81885 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:33:59.885667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f729c-5398-4895-b4a8-45fe30c81885 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:33:59.896527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 211/407] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:33:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:33:59.901747 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f729c-5398-4895-b4a8-45fe30c81885 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:33:59.930191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897f636-915b-4ef1-b9e1-b09def9663da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:33:59.930759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897f636-915b-4ef1-b9e1-b09def9663da None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:33:59.931036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897f636-915b-4ef1-b9e1-b09def9663da None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:33:59.931384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897f636-915b-4ef1-b9e1-b09def9663da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:33:59.953080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3897f636-915b-4ef1-b9e1-b09def9663da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:33:59.956768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-3897f636-915b-4ef1-b9e1-b09def9663da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:33:59.959882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:33:59.960751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:33:59.962196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:33:59.962196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:33:59.962196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:33:59.962196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:33:59.962196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:33:59.962196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:33:59.962196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:33:59.962196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:33:59.962196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:33:59.962196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:33:59.962196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:33:59.962196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:33:59.962196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 197/408] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:33:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:33:59.970833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897f636-915b-4ef1-b9e1-b09def9663da None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:00.043062 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f729c-5398-4895-b4a8-45fe30c81885 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:00.086714 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145d46b-c87f-46a2-a57e-f840f106c94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:00.087358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145d46b-c87f-46a2-a57e-f840f106c94e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:00.087629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145d46b-c87f-46a2-a57e-f840f106c94e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:00.087894 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145d46b-c87f-46a2-a57e-f840f106c94e None admin] PATH_INFO: `/v3/groups/0224dbb2925e4a2b99e29e07cefccd01` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:00.088993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145d46b-c87f-46a2-a57e-f840f106c94e None admin] RBAC: Authorizing `identity:get_group(group_id=0224dbb2925e4a2b99e29e07cefccd01)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:00.093438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145d46b-c87f-46a2-a57e-f840f106c94e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:00.095481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 212/409] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:33:59 2022] GET /identity/v3/groups/0224dbb2925e4a2b99e29e07cefccd01 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:00.101587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145d46b-c87f-46a2-a57e-f840f106c94e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:00.129145 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897f636-915b-4ef1-b9e1-b09def9663da None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:00.209331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bf6ec-7f18-4512-9b24-f63346a31c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:00.209906 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bf6ec-7f18-4512-9b24-f63346a31c7c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:00.210282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bf6ec-7f18-4512-9b24-f63346a31c7c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:00.210559 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9bf6ec-7f18-4512-9b24-f63346a31c7c None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:00.211228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9bf6ec-7f18-4512-9b24-f63346a31c7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:00.211532 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9bf6ec-7f18-4512-9b24-f63346a31c7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:00.213038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9bf6ec-7f18-4512-9b24-f63346a31c7c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:00.229127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:00.232316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145d46b-c87f-46a2-a57e-f840f106c94e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:00.235866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 198/410] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:33:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:00.241061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bf6ec-7f18-4512-9b24-f63346a31c7c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:00.269129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9cbfe-0165-46da-b663-22ad61116f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:00.269767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9cbfe-0165-46da-b663-22ad61116f0e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:00.270078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9cbfe-0165-46da-b663-22ad61116f0e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:00.270352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f9cbfe-0165-46da-b663-22ad61116f0e None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:00.271459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9cbfe-0165-46da-b663-22ad61116f0e None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:00.272904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f9cbfe-0165-46da-b663-22ad61116f0e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:00.274982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 213/411] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:34:00 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:00.286694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9cbfe-0165-46da-b663-22ad61116f0e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:00.295204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:00.398403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9bf6ec-7f18-4512-9b24-f63346a31c7c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:00.462212 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f9cbfe-0165-46da-b663-22ad61116f0e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:00.465704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf870b-f869-4785-929b-924256068b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:00.466494 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf870b-f869-4785-929b-924256068b2a None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:00.466773 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf870b-f869-4785-929b-924256068b2a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:00.467038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baf870b-f869-4785-929b-924256068b2a None admin] PATH_INFO: `/v3/groups/36c71cc3f53742a99696232600eae5eb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:00.472741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf870b-f869-4785-929b-924256068b2a None admin] RBAC: Authorizing `identity:get_group(group_id=36c71cc3f53742a99696232600eae5eb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:00.474331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baf870b-f869-4785-929b-924256068b2a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:00.476433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 199/412] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:34:00 2022] GET /identity/v3/groups/36c71cc3f53742a99696232600eae5eb => generated 229 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:00.482725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf870b-f869-4785-929b-924256068b2a None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:00.519846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912afddb-8357-411b-a919-4b8a90dec4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:00.520567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912afddb-8357-411b-a919-4b8a90dec4f7 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:00.520844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912afddb-8357-411b-a919-4b8a90dec4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:00.521111 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912afddb-8357-411b-a919-4b8a90dec4f7 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552/groups/0224dbb2925e4a2b99e29e07cefccd01/roles/36b107b4892f47f8af6b65e1d1d94bb0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:00.526754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912afddb-8357-411b-a919-4b8a90dec4f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f5fbdcb3c4884d59ab798fc1c9752552, group_id=0224dbb2925e4a2b99e29e07cefccd01, role_id=36b107b4892f47f8af6b65e1d1d94bb0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:00.528195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912afddb-8357-411b-a919-4b8a90dec4f7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:00.612022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:00.621461 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 214/413] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Sep 14 19:34:00 2022] PUT /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552/groups/0224dbb2925e4a2b99e29e07cefccd01/roles/36b107b4892f47f8af6b65e1d1d94bb0 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:34:00.622284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baf870b-f869-4785-929b-924256068b2a None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:00.623544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fbb7d-91e0-4a09-bb29-85a82447b075 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:00.623892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fbb7d-91e0-4a09-bb29-85a82447b075 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:00.624232 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fbb7d-91e0-4a09-bb29-85a82447b075 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:00.630048 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 215/414] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:00.632318 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c4673-15e1-46ba-8660-ba0fb8aab483 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:00.632830 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c4673-15e1-46ba-8660-ba0fb8aab483 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:00.633306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c4673-15e1-46ba-8660-ba0fb8aab483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:00.655114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-441c4673-15e1-46ba-8660-ba0fb8aab483 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:00.656690 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-441c4673-15e1-46ba-8660-ba0fb8aab483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:00.666632 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55a384-2f31-47c2-bde1-f87d6daaf7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:00.667288 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55a384-2f31-47c2-bde1-f87d6daaf7c3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:00.667567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55a384-2f31-47c2-bde1-f87d6daaf7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:00.667837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55a384-2f31-47c2-bde1-f87d6daaf7c3 None admin] PATH_INFO: `/v3/projects/2a8f053b53ae4aff97a668669e77d899` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:00.669094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55a384-2f31-47c2-bde1-f87d6daaf7c3 None admin] RBAC: Authorizing `identity:get_project(project_id=2a8f053b53ae4aff97a668669e77d899)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:00.671285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55a384-2f31-47c2-bde1-f87d6daaf7c3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:00.673559 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 200/415] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:34:00 2022] GET /identity/v3/projects/2a8f053b53ae4aff97a668669e77d899 => generated 344 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:00.679319 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55a384-2f31-47c2-bde1-f87d6daaf7c3 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:00.687086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:00.746121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 216/416] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:34:00.868630 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55a384-2f31-47c2-bde1-f87d6daaf7c3 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:00.902852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac99fe-bfc1-4ff7-a7cc-6a2e167b074f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:00.903497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac99fe-bfc1-4ff7-a7cc-6a2e167b074f None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:00.903771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac99fe-bfc1-4ff7-a7cc-6a2e167b074f None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:00.904065 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac99fe-bfc1-4ff7-a7cc-6a2e167b074f None admin] PATH_INFO: `/v3/projects/2a8f053b53ae4aff97a668669e77d899/groups/36c71cc3f53742a99696232600eae5eb/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:00.905999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac99fe-bfc1-4ff7-a7cc-6a2e167b074f None admin] RBAC: Authorizing `identity:create_grant(project_id=2a8f053b53ae4aff97a668669e77d899, group_id=36c71cc3f53742a99696232600eae5eb, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:00.907438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfac99fe-bfc1-4ff7-a7cc-6a2e167b074f None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:00.980894 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 201/417] 10.222.0.42 () {58 vars in 1442 bytes} [Wed Sep 14 19:34:00 2022] PUT /identity/v3/projects/2a8f053b53ae4aff97a668669e77d899/groups/36c71cc3f53742a99696232600eae5eb/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:34:01.100242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c4673-15e1-46ba-8660-ba0fb8aab483 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:01.219968 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c4673-15e1-46ba-8660-ba0fb8aab483 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:01.258658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a9b30-6640-482e-b497-c481e452fab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:01.259554 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a9b30-6640-482e-b497-c481e452fab5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:01.259995 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a9b30-6640-482e-b497-c481e452fab5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:01.260424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56a9b30-6640-482e-b497-c481e452fab5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:01.272044 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56a9b30-6640-482e-b497-c481e452fab5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-c56a9b30-6640-482e-b497-c481e452fab5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:01.276892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:01.277744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:01.278582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:01.278582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:01.278582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:01.278582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:01.278582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:01.278582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:01.278582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:01.278582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:01.278582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:01.278582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:01.278582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:34:01.278582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:01.278582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 217/418] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:01.283847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac99fe-bfc1-4ff7-a7cc-6a2e167b074f None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:01.313638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:01.397530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfac99fe-bfc1-4ff7-a7cc-6a2e167b074f None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:01.440478 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d0dd2-42d8-42b1-8fb5-80029601f428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:01.441088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d0dd2-42d8-42b1-8fb5-80029601f428 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:01.441370 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d0dd2-42d8-42b1-8fb5-80029601f428 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:01.441643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053d0dd2-42d8-42b1-8fb5-80029601f428 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:01.442634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d0dd2-42d8-42b1-8fb5-80029601f428 None admin] RBAC: Adding query filter params (name=member) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:01.442941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d0dd2-42d8-42b1-8fb5-80029601f428 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:01.444409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053d0dd2-42d8-42b1-8fb5-80029601f428 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:01.459543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 202/419] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:34:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:01.464967 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a9b30-6640-482e-b497-c481e452fab5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:01.489727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:01.574500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56a9b30-6640-482e-b497-c481e452fab5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:01.610421 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017cc65-4605-46ad-b1b9-8e3bbbb2c731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:01.611029 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017cc65-4605-46ad-b1b9-8e3bbbb2c731 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:01.611307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017cc65-4605-46ad-b1b9-8e3bbbb2c731 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:01.611578 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017cc65-4605-46ad-b1b9-8e3bbbb2c731 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:01.680598 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017cc65-4605-46ad-b1b9-8e3bbbb2c731 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:01.682019 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017cc65-4605-46ad-b1b9-8e3bbbb2c731 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:01.700785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:01.786149 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 218/420] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:34:01.795072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d0dd2-42d8-42b1-8fb5-80029601f428 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:01.907944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:01.923717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053d0dd2-42d8-42b1-8fb5-80029601f428 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:01.971543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3dce0-794f-4c52-a81f-e14ca2979325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:01.972107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3dce0-794f-4c52-a81f-e14ca2979325 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:01.972386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3dce0-794f-4c52-a81f-e14ca2979325 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:01.972661 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3dce0-794f-4c52-a81f-e14ca2979325 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:01.973881 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3dce0-794f-4c52-a81f-e14ca2979325 None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:01.975424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c3dce0-794f-4c52-a81f-e14ca2979325 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:01.977062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 203/421] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:34:01 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:01.982756 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017cc65-4605-46ad-b1b9-8e3bbbb2c731 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:02.088183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017cc65-4605-46ad-b1b9-8e3bbbb2c731 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:02.106077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:02.124202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b41639-bd22-4e0b-ac11-50b009dc8396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:02.124868 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b41639-bd22-4e0b-ac11-50b009dc8396 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:02.125148 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b41639-bd22-4e0b-ac11-50b009dc8396 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:02.125424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b41639-bd22-4e0b-ac11-50b009dc8396 None admin] PATH_INFO: `/v3/groups/0224dbb2925e4a2b99e29e07cefccd01` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:02.126698 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b41639-bd22-4e0b-ac11-50b009dc8396 None admin] RBAC: Authorizing `identity:get_group(group_id=0224dbb2925e4a2b99e29e07cefccd01)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:02.128188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b41639-bd22-4e0b-ac11-50b009dc8396 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:02.130348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 219/422] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:34:01 2022] GET /identity/v3/groups/0224dbb2925e4a2b99e29e07cefccd01 => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:02.139933 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3dce0-794f-4c52-a81f-e14ca2979325 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:02.254701 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303819b0-e180-44e5-b83e-771b70246c6f None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:02.255022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303819b0-e180-44e5-b83e-771b70246c6f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:02.255215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303819b0-e180-44e5-b83e-771b70246c6f None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:02.261957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 220/423] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:02.273477 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113b600-2576-4ccf-ae4f-20358f850661 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:02.273787 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113b600-2576-4ccf-ae4f-20358f850661 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:02.274056 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113b600-2576-4ccf-ae4f-20358f850661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:02.279028 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3dce0-794f-4c52-a81f-e14ca2979325 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:02.314854 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e113b600-2576-4ccf-ae4f-20358f850661 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:02.316038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bde23c-04f1-437c-bd16-5fc796728cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:02.316485 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e113b600-2576-4ccf-ae4f-20358f850661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:02.316672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bde23c-04f1-437c-bd16-5fc796728cdd None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:02.316925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bde23c-04f1-437c-bd16-5fc796728cdd None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:02.317202 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bde23c-04f1-437c-bd16-5fc796728cdd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:02.318214 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bde23c-04f1-437c-bd16-5fc796728cdd None admin] RBAC: Adding query filter params (group.id=0224dbb2925e4a2b99e29e07cefccd01, role.id=efa4ca02b2ff4ad6b14d501d4d79bbea, scope.project.id=07768f903a224486a253fc7acc9c3360) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:02.318524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bde23c-04f1-437c-bd16-5fc796728cdd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:02.319967 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bde23c-04f1-437c-bd16-5fc796728cdd None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:02.327249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 204/424] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 19:34:02 2022] GET /identity/v3/role_assignments?group.id=0224dbb2925e4a2b99e29e07cefccd01&scope.project.id=07768f903a224486a253fc7acc9c3360&role.id=efa4ca02b2ff4ad6b14d501d4d79bbea => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:02.433427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 221/425] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:02.440635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f78573-c050-4ce0-8e03-49482c606e2c None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:02.440947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f78573-c050-4ce0-8e03-49482c606e2c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:02.441191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f78573-c050-4ce0-8e03-49482c606e2c None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:02.443189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 205/426] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:02.449070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b099c-9cec-4d62-acfe-906f82969b78 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:02.449400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b099c-9cec-4d62-acfe-906f82969b78 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:02.449617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b099c-9cec-4d62-acfe-906f82969b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:02.475445 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-702b099c-9cec-4d62-acfe-906f82969b78 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:02.476717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-702b099c-9cec-4d62-acfe-906f82969b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:02.576372 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 222/427] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:02.715425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f78573-c050-4ce0-8e03-49482c606e2c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:02.735691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:02.832423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f78573-c050-4ce0-8e03-49482c606e2c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:02.877022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27688e5b-7df7-4109-968f-5478d9fbcaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:02.877883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27688e5b-7df7-4109-968f-5478d9fbcaf2 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:02.878310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27688e5b-7df7-4109-968f-5478d9fbcaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:02.878723 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27688e5b-7df7-4109-968f-5478d9fbcaf2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:02.890248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27688e5b-7df7-4109-968f-5478d9fbcaf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-27688e5b-7df7-4109-968f-5478d9fbcaf2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:02.895736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:02.896803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:02.897813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:02.897813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:02.897813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:02.897813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:02.897813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:02.897813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:02.897813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:02.897813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:02.897813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:02.897813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:02.897813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 19:34:02.897813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:02.902942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 206/428] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:02.904056 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b099c-9cec-4d62-acfe-906f82969b78 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:02.905597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27688e5b-7df7-4109-968f-5478d9fbcaf2 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:02.979347 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:03.007720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:03.041373 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27688e5b-7df7-4109-968f-5478d9fbcaf2 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:03.052457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b099c-9cec-4d62-acfe-906f82969b78 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:03.089131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b686e5a-4cb1-4a07-84e9-5c415c1d83aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:03.090194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b686e5a-4cb1-4a07-84e9-5c415c1d83aa None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:03.090734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b686e5a-4cb1-4a07-84e9-5c415c1d83aa None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:03.091276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b686e5a-4cb1-4a07-84e9-5c415c1d83aa None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:03.092129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b686e5a-4cb1-4a07-84e9-5c415c1d83aa None admin] RBAC: Adding query filter params (name=member) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:03.092576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b686e5a-4cb1-4a07-84e9-5c415c1d83aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:03.094305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b686e5a-4cb1-4a07-84e9-5c415c1d83aa None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:03.094722 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ade32-fbd6-40d3-adcd-8b9eecc18f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:03.095651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ade32-fbd6-40d3-adcd-8b9eecc18f06 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:03.096141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ade32-fbd6-40d3-adcd-8b9eecc18f06 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:03.096611 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0ade32-fbd6-40d3-adcd-8b9eecc18f06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:03.104173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 207/429] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:34:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:03.108614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0ade32-fbd6-40d3-adcd-8b9eecc18f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:03.110175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b686e5a-4cb1-4a07-84e9-5c415c1d83aa None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-5f0ade32-fbd6-40d3-adcd-8b9eecc18f06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:03.114319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:03.115196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:03.115969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:03.115969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:03.115969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:03.115969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:03.115969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:03.115969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:03.115969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:03.115969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:03.115969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:03.115969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:03.115969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:34:03.115969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:03.119484 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 223/430] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:34:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:03.127461 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ade32-fbd6-40d3-adcd-8b9eecc18f06 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:03.248907 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b686e5a-4cb1-4a07-84e9-5c415c1d83aa None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:03.250043 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0ade32-fbd6-40d3-adcd-8b9eecc18f06 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:03.258813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:03.299980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e22600-4b60-42c0-a13e-07fbdfe19533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:03.300416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b437ba8-bdd9-49a0-8b5c-8afac2013294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:03.301051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b437ba8-bdd9-49a0-8b5c-8afac2013294 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:03.301299 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e22600-4b60-42c0-a13e-07fbdfe19533 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:03.301409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b437ba8-bdd9-49a0-8b5c-8afac2013294 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:03.301587 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b437ba8-bdd9-49a0-8b5c-8afac2013294 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:03.302020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e22600-4b60-42c0-a13e-07fbdfe19533 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:03.302561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e22600-4b60-42c0-a13e-07fbdfe19533 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:03.303389 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e22600-4b60-42c0-a13e-07fbdfe19533 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:03.303905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e22600-4b60-42c0-a13e-07fbdfe19533 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:03.305700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e22600-4b60-42c0-a13e-07fbdfe19533 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:03.312925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:03.323610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 224/431] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:03.324799 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e22600-4b60-42c0-a13e-07fbdfe19533 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:03.397623 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b437ba8-bdd9-49a0-8b5c-8afac2013294 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:03.399481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b437ba8-bdd9-49a0-8b5c-8afac2013294 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:03.445085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e22600-4b60-42c0-a13e-07fbdfe19533 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:03.480224 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8f9431-38df-4cba-981c-83cb5e0ee254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:03.480971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f9431-38df-4cba-981c-83cb5e0ee254 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:03.481160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f9431-38df-4cba-981c-83cb5e0ee254 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:03.481381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8f9431-38df-4cba-981c-83cb5e0ee254 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:03.519362 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:03.521235 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 208/432] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 19:34:03.522629 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497e4bc-a5f5-45f7-b22e-bf2d553a4f3c None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:03.522915 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497e4bc-a5f5-45f7-b22e-bf2d553a4f3c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:03.523184 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497e4bc-a5f5-45f7-b22e-bf2d553a4f3c None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:03.524435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 209/433] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:03.529479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497e4bc-a5f5-45f7-b22e-bf2d553a4f3c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:03.556718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f9431-38df-4cba-981c-83cb5e0ee254 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:03.558205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8f9431-38df-4cba-981c-83cb5e0ee254 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:03.572867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:03.632683 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497e4bc-a5f5-45f7-b22e-bf2d553a4f3c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:03.662906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 225/434] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:34:03.664457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7d9c3-2777-4101-888e-0899213f15d8 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:03.664940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7d9c3-2777-4101-888e-0899213f15d8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:03.665430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7d9c3-2777-4101-888e-0899213f15d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:03.675133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124ba53-601a-44eb-b6c5-e02cd5e48390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:03.675816 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124ba53-601a-44eb-b6c5-e02cd5e48390 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:03.676096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124ba53-601a-44eb-b6c5-e02cd5e48390 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:03.676367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124ba53-601a-44eb-b6c5-e02cd5e48390 None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:03.677498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124ba53-601a-44eb-b6c5-e02cd5e48390 None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:03.679028 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5124ba53-601a-44eb-b6c5-e02cd5e48390 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:03.683741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-25b7d9c3-2777-4101-888e-0899213f15d8 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:03.685173 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-25b7d9c3-2777-4101-888e-0899213f15d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:03.685872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 210/435] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:34:03 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:03.686640 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124ba53-601a-44eb-b6c5-e02cd5e48390 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:03.749794 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:03.755675 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 226/436] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:03.756496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7d9c3-2777-4101-888e-0899213f15d8 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:03.802819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124ba53-601a-44eb-b6c5-e02cd5e48390 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:03.811643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:03.849085 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa404e2-1d04-454e-a1e4-4a1a399476b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:03.849849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa404e2-1d04-454e-a1e4-4a1a399476b4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:03.850130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa404e2-1d04-454e-a1e4-4a1a399476b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:03.850392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa404e2-1d04-454e-a1e4-4a1a399476b4 None admin] PATH_INFO: `/v3/projects/07768f903a224486a253fc7acc9c3360` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:03.851546 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa404e2-1d04-454e-a1e4-4a1a399476b4 None admin] RBAC: Authorizing `identity:get_project(project_id=07768f903a224486a253fc7acc9c3360)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:03.853097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa404e2-1d04-454e-a1e4-4a1a399476b4 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:03.862525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 211/437] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:34:03 2022] GET /identity/v3/projects/07768f903a224486a253fc7acc9c3360 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:03.863927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa404e2-1d04-454e-a1e4-4a1a399476b4 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:03.885100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7d9c3-2777-4101-888e-0899213f15d8 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:03.922430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc2361-3415-4086-b1d6-415c6ba90a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:03.923385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc2361-3415-4086-b1d6-415c6ba90a77 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:03.923866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc2361-3415-4086-b1d6-415c6ba90a77 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:03.924370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc2361-3415-4086-b1d6-415c6ba90a77 None admin] PATH_INFO: `/v3/groups/0224dbb2925e4a2b99e29e07cefccd01` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:03.925772 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc2361-3415-4086-b1d6-415c6ba90a77 None admin] RBAC: Authorizing `identity:get_group(group_id=0224dbb2925e4a2b99e29e07cefccd01)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:03.927679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc2361-3415-4086-b1d6-415c6ba90a77 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:03.929290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 227/438] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:34:03 2022] GET /identity/v3/groups/0224dbb2925e4a2b99e29e07cefccd01 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:03.931143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426d8c4-1c08-4079-a0f2-e8c27dc10338 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:03.931703 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426d8c4-1c08-4079-a0f2-e8c27dc10338 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:03.932290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426d8c4-1c08-4079-a0f2-e8c27dc10338 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:03.933883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 228/439] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:03.938568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426d8c4-1c08-4079-a0f2-e8c27dc10338 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:03.980957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa404e2-1d04-454e-a1e4-4a1a399476b4 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:04.002986 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:04.018459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92113b7-58cf-4072-8524-3d279e1ec10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:04.019139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92113b7-58cf-4072-8524-3d279e1ec10f None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:04.019420 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92113b7-58cf-4072-8524-3d279e1ec10f None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:04.019693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92113b7-58cf-4072-8524-3d279e1ec10f None admin] PATH_INFO: `/v3/groups/0224dbb2925e4a2b99e29e07cefccd01` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:04.020849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92113b7-58cf-4072-8524-3d279e1ec10f None admin] RBAC: Authorizing `identity:get_group(group_id=0224dbb2925e4a2b99e29e07cefccd01)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:04.022373 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92113b7-58cf-4072-8524-3d279e1ec10f None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:04.029523 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 212/440] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:34:03 2022] GET /identity/v3/groups/0224dbb2925e4a2b99e29e07cefccd01 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:04.029523 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42630e-2266-40c4-b8a0-e15b395f5b9a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:04.029523 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42630e-2266-40c4-b8a0-e15b395f5b9a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:04.029523 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a42630e-2266-40c4-b8a0-e15b395f5b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:04.043056 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426d8c4-1c08-4079-a0f2-e8c27dc10338 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:04.045964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-9a42630e-2266-40c4-b8a0-e15b395f5b9a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:04.047152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-9a42630e-2266-40c4-b8a0-e15b395f5b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:04.077181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f22d9-63aa-4c4c-a171-87239218a696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:04.077879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f22d9-63aa-4c4c-a171-87239218a696 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:04.078163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f22d9-63aa-4c4c-a171-87239218a696 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:04.078434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0f22d9-63aa-4c4c-a171-87239218a696 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:04.079147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f22d9-63aa-4c4c-a171-87239218a696 None admin] RBAC: Adding query filter params (group.id=0224dbb2925e4a2b99e29e07cefccd01, role.id=efa4ca02b2ff4ad6b14d501d4d79bbea, scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:04.079453 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f22d9-63aa-4c4c-a171-87239218a696 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:04.080867 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0f22d9-63aa-4c4c-a171-87239218a696 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:04.081538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:04.086578 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 229/441] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 19:34:03 2022] GET /identity/v3/role_assignments?group.id=0224dbb2925e4a2b99e29e07cefccd01&scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552&role.id=efa4ca02b2ff4ad6b14d501d4d79bbea => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:04.089319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f22d9-63aa-4c4c-a171-87239218a696 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:04.116024 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 213/442] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:04.117107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42630e-2266-40c4-b8a0-e15b395f5b9a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:04.227012 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0f22d9-63aa-4c4c-a171-87239218a696 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:04.241252 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a42630e-2266-40c4-b8a0-e15b395f5b9a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:04.261912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:04.268797 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdbca5-cd39-4c99-8318-22997c1e1b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:04.269601 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdbca5-cd39-4c99-8318-22997c1e1b9e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:04.270067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdbca5-cd39-4c99-8318-22997c1e1b9e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:04.274023 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdbca5-cd39-4c99-8318-22997c1e1b9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:04.274023 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdbca5-cd39-4c99-8318-22997c1e1b9e None admin] RBAC: Adding query filter params (group.id=0224dbb2925e4a2b99e29e07cefccd01, role.id=36b107b4892f47f8af6b65e1d1d94bb0, scope.project.id=07768f903a224486a253fc7acc9c3360) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:04.274023 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdbca5-cd39-4c99-8318-22997c1e1b9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:04.274023 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdbca5-cd39-4c99-8318-22997c1e1b9e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:04.282397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 230/443] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 19:34:04 2022] GET /identity/v3/role_assignments?group.id=0224dbb2925e4a2b99e29e07cefccd01&scope.project.id=07768f903a224486a253fc7acc9c3360&role.id=36b107b4892f47f8af6b65e1d1d94bb0 => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:04.289439 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca996566-5697-486e-8918-4aa0d849c408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:04.290822 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca996566-5697-486e-8918-4aa0d849c408 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:04.291443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca996566-5697-486e-8918-4aa0d849c408 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:04.292053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca996566-5697-486e-8918-4aa0d849c408 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:04.305842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca996566-5697-486e-8918-4aa0d849c408 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-ca996566-5697-486e-8918-4aa0d849c408 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:04.311611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:04.312754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:04.313743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:04.313743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:04.313743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:04.313743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:04.313743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:04.313743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:04.313743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:04.313743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:04.313743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:04.313743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:04.313743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:34:04.313743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:04.315868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 214/444] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:34:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:04.322355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdbca5-cd39-4c99-8318-22997c1e1b9e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:04.417175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca996566-5697-486e-8918-4aa0d849c408 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:04.436502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdbca5-cd39-4c99-8318-22997c1e1b9e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:04.450546 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:04.477488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69157a-df31-460b-8a51-8cb1b131eed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:04.478206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69157a-df31-460b-8a51-8cb1b131eed7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:04.478491 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69157a-df31-460b-8a51-8cb1b131eed7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:04.478765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69157a-df31-460b-8a51-8cb1b131eed7 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:04.479420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69157a-df31-460b-8a51-8cb1b131eed7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:04.479725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69157a-df31-460b-8a51-8cb1b131eed7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:04.481142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69157a-df31-460b-8a51-8cb1b131eed7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:04.491713 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 231/445] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:04.497262 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69157a-df31-460b-8a51-8cb1b131eed7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:04.527890 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca996566-5697-486e-8918-4aa0d849c408 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:04.562197 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:04.568683 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e513e-0943-441f-9ec9-7c8bc3455793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:04.569374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e513e-0943-441f-9ec9-7c8bc3455793 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:04.569645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e513e-0943-441f-9ec9-7c8bc3455793 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:04.569997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e513e-0943-441f-9ec9-7c8bc3455793 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:04.581588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e513e-0943-441f-9ec9-7c8bc3455793 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-0f2e513e-0943-441f-9ec9-7c8bc3455793 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:04.586592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:04.587477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:04.588289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:04.588289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:04.588289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:04.588289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:04.588289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:04.588289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:04.588289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:04.588289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:04.588289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:04.588289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:04.588289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:34:04.588289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:04.588289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 215/446] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:34:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:04.595567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e513e-0943-441f-9ec9-7c8bc3455793 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:04.615923 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69157a-df31-460b-8a51-8cb1b131eed7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:04.663241 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dc681-71dc-4783-bccb-66e0f293bc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:04.663860 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dc681-71dc-4783-bccb-66e0f293bc39 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:04.664138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dc681-71dc-4783-bccb-66e0f293bc39 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:04.664405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833dc681-71dc-4783-bccb-66e0f293bc39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:04.701322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:04.710832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e513e-0943-441f-9ec9-7c8bc3455793 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:04.742419 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833dc681-71dc-4783-bccb-66e0f293bc39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:04.743864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833dc681-71dc-4783-bccb-66e0f293bc39 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:04.754582 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4d461-89e6-49e6-92e0-9ad6afd8efc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:04.755983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4d461-89e6-49e6-92e0-9ad6afd8efc2 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:04.755983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4d461-89e6-49e6-92e0-9ad6afd8efc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:04.755983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4d461-89e6-49e6-92e0-9ad6afd8efc2 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:04.756345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4d461-89e6-49e6-92e0-9ad6afd8efc2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:04.756639 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4d461-89e6-49e6-92e0-9ad6afd8efc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:04.758062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4d461-89e6-49e6-92e0-9ad6afd8efc2 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:04.768848 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 216/447] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:04.774495 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4d461-89e6-49e6-92e0-9ad6afd8efc2 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:04.811075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:04.863047 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 232/448] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:34:04.868535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dc681-71dc-4783-bccb-66e0f293bc39 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:04.888954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4d461-89e6-49e6-92e0-9ad6afd8efc2 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:04.920968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302780d-f4de-44d0-9173-7a0b3a792978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:04.921568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302780d-f4de-44d0-9173-7a0b3a792978 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:04.921891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302780d-f4de-44d0-9173-7a0b3a792978 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:04.922167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a302780d-f4de-44d0-9173-7a0b3a792978 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:04.945991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:04.966908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833dc681-71dc-4783-bccb-66e0f293bc39 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:04.988147 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302780d-f4de-44d0-9173-7a0b3a792978 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:04.989609 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a302780d-f4de-44d0-9173-7a0b3a792978 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:05.000096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36593bfe-83bc-4fd2-93c1-326a4ffa1f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:05.000665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36593bfe-83bc-4fd2-93c1-326a4ffa1f5e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:05.000943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36593bfe-83bc-4fd2-93c1-326a4ffa1f5e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:05.001216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36593bfe-83bc-4fd2-93c1-326a4ffa1f5e None admin] PATH_INFO: `/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:05.002354 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36593bfe-83bc-4fd2-93c1-326a4ffa1f5e None admin] RBAC: Authorizing `identity:get_project(project_id=f5fbdcb3c4884d59ab798fc1c9752552)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:05.003776 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36593bfe-83bc-4fd2-93c1-326a4ffa1f5e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:05.005377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 233/449] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:34:04 2022] GET /identity/v3/projects/f5fbdcb3c4884d59ab798fc1c9752552 => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:05.010773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36593bfe-83bc-4fd2-93c1-326a4ffa1f5e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:05.034878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:05.085850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 217/450] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 19:34:05.091884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302780d-f4de-44d0-9173-7a0b3a792978 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:05.107105 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36593bfe-83bc-4fd2-93c1-326a4ffa1f5e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:05.139558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9460b-22a2-4454-a721-cc113ebc3b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:05.140197 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9460b-22a2-4454-a721-cc113ebc3b4c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:05.140474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9460b-22a2-4454-a721-cc113ebc3b4c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:05.140746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9460b-22a2-4454-a721-cc113ebc3b4c None admin] PATH_INFO: `/v3/groups/0224dbb2925e4a2b99e29e07cefccd01` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:05.141852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9460b-22a2-4454-a721-cc113ebc3b4c None admin] RBAC: Authorizing `identity:get_group(group_id=0224dbb2925e4a2b99e29e07cefccd01)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:05.143271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9460b-22a2-4454-a721-cc113ebc3b4c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:05.143387 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:05.144800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 234/451] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:34:05 2022] GET /identity/v3/groups/0224dbb2925e4a2b99e29e07cefccd01 => generated 226 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:05.150516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9460b-22a2-4454-a721-cc113ebc3b4c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:05.186088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a302780d-f4de-44d0-9173-7a0b3a792978 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:05.218369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f390b-000c-4d71-9edc-19b3254d30f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:05.218980 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f390b-000c-4d71-9edc-19b3254d30f3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:05.219253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f390b-000c-4d71-9edc-19b3254d30f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:05.219515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f390b-000c-4d71-9edc-19b3254d30f3 None admin] PATH_INFO: `/v3/projects/2a8f053b53ae4aff97a668669e77d899` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:05.220605 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f390b-000c-4d71-9edc-19b3254d30f3 None admin] RBAC: Authorizing `identity:get_project(project_id=2a8f053b53ae4aff97a668669e77d899)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:05.222074 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f390b-000c-4d71-9edc-19b3254d30f3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:05.223663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 218/452] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:34:05 2022] GET /identity/v3/projects/2a8f053b53ae4aff97a668669e77d899 => generated 344 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:05.229091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f390b-000c-4d71-9edc-19b3254d30f3 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:05.239156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:05.247369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9460b-22a2-4454-a721-cc113ebc3b4c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:05.280676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff06bb-1ed4-41f8-ac3a-1ccb9bd452f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:05.281231 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff06bb-1ed4-41f8-ac3a-1ccb9bd452f5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:05.281504 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff06bb-1ed4-41f8-ac3a-1ccb9bd452f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:05.281768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff06bb-1ed4-41f8-ac3a-1ccb9bd452f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:05.282513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff06bb-1ed4-41f8-ac3a-1ccb9bd452f5 None admin] RBAC: Adding query filter params (group.id=0224dbb2925e4a2b99e29e07cefccd01, role.id=36b107b4892f47f8af6b65e1d1d94bb0, scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:05.282812 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff06bb-1ed4-41f8-ac3a-1ccb9bd452f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:05.284336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff06bb-1ed4-41f8-ac3a-1ccb9bd452f5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:05.290129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 235/453] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 19:34:05 2022] GET /identity/v3/role_assignments?group.id=0224dbb2925e4a2b99e29e07cefccd01&scope.project.id=f5fbdcb3c4884d59ab798fc1c9752552&role.id=36b107b4892f47f8af6b65e1d1d94bb0 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:05.331705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f390b-000c-4d71-9edc-19b3254d30f3 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:05.356367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:05.368321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa8ed8-2894-4dac-a9eb-560cb48d0371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:05.368956 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa8ed8-2894-4dac-a9eb-560cb48d0371 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:05.369232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa8ed8-2894-4dac-a9eb-560cb48d0371 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:05.369503 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa8ed8-2894-4dac-a9eb-560cb48d0371 None admin] PATH_INFO: `/v3/groups/36c71cc3f53742a99696232600eae5eb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:05.370683 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa8ed8-2894-4dac-a9eb-560cb48d0371 None admin] RBAC: Authorizing `identity:get_group(group_id=36c71cc3f53742a99696232600eae5eb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:05.372130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa8ed8-2894-4dac-a9eb-560cb48d0371 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:05.374394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 219/454] 10.222.0.42 () {56 vars in 1180 bytes} [Wed Sep 14 19:34:05 2022] GET /identity/v3/groups/36c71cc3f53742a99696232600eae5eb => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:05.380606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff06bb-1ed4-41f8-ac3a-1ccb9bd452f5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:05.481598 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff06bb-1ed4-41f8-ac3a-1ccb9bd452f5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:05.515027 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011e767-90cb-4b74-b5fe-e82133cfb061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:05.515630 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011e767-90cb-4b74-b5fe-e82133cfb061 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:05.515900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011e767-90cb-4b74-b5fe-e82133cfb061 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:05.516166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011e767-90cb-4b74-b5fe-e82133cfb061 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:05.516871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011e767-90cb-4b74-b5fe-e82133cfb061 None admin] RBAC: Adding query filter params (group.id=36c71cc3f53742a99696232600eae5eb, role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=2a8f053b53ae4aff97a668669e77d899) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:05.517169 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011e767-90cb-4b74-b5fe-e82133cfb061 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:05.518616 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2011e767-90cb-4b74-b5fe-e82133cfb061 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:05.524283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 236/455] 10.222.0.42 () {56 vars in 1376 bytes} [Wed Sep 14 19:34:05 2022] GET /identity/v3/role_assignments?group.id=36c71cc3f53742a99696232600eae5eb&scope.project.id=2a8f053b53ae4aff97a668669e77d899&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:08.017978 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9266414-bc3a-43df-9f90-7f78c5140cb0 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:08.018505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9266414-bc3a-43df-9f90-7f78c5140cb0 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:08.018505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9266414-bc3a-43df-9f90-7f78c5140cb0 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:08.020132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 220/456] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:08.025160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c176df6-ea4f-4e76-84cc-e4f566f3f553 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:08.025539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c176df6-ea4f-4e76-84cc-e4f566f3f553 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:08.025700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c176df6-ea4f-4e76-84cc-e4f566f3f553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:08.050771 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2c176df6-ea4f-4e76-84cc-e4f566f3f553 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:08.051972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2c176df6-ea4f-4e76-84cc-e4f566f3f553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:08.074688 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:08.136221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 237/457] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:08.280544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc809ac-8cf4-45af-952e-ec696571de23 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:08.280544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc809ac-8cf4-45af-952e-ec696571de23 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:08.280789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc809ac-8cf4-45af-952e-ec696571de23 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:08.282400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 221/458] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:08.287405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e826d90-f7e8-4417-af49-1e30ef210130 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:08.287687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e826d90-f7e8-4417-af49-1e30ef210130 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:08.287961 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e826d90-f7e8-4417-af49-1e30ef210130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:08.316959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7e826d90-f7e8-4417-af49-1e30ef210130 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:08.318434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7e826d90-f7e8-4417-af49-1e30ef210130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:08.404727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 238/459] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:08.511959 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc809ac-8cf4-45af-952e-ec696571de23 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:08.649135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc809ac-8cf4-45af-952e-ec696571de23 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:08.699560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00e59cf-97e8-41cb-9271-8dac02ebec8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:08.700112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e59cf-97e8-41cb-9271-8dac02ebec8f None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:08.700392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e59cf-97e8-41cb-9271-8dac02ebec8f None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:08.700721 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00e59cf-97e8-41cb-9271-8dac02ebec8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:08.714466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e59cf-97e8-41cb-9271-8dac02ebec8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:08.715923 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00e59cf-97e8-41cb-9271-8dac02ebec8f None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-a00e59cf-97e8-41cb-9271-8dac02ebec8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:08.735466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:08.736440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:08.737346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:08.737346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:08.737346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:08.737346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 222/460] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:08.743103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e826d90-f7e8-4417-af49-1e30ef210130 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:08.760022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d580e-3350-4b84-8648-68e82c6337dd None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:08.760383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d580e-3350-4b84-8648-68e82c6337dd None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:08.760566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d580e-3350-4b84-8648-68e82c6337dd None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:08.762314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 223/461] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:08.768381 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae845bf-cf6c-45a4-bb8b-24c80dd95203 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:08.768672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae845bf-cf6c-45a4-bb8b-24c80dd95203 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:08.768939 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae845bf-cf6c-45a4-bb8b-24c80dd95203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:08.790354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-4ae845bf-cf6c-45a4-bb8b-24c80dd95203 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:08.791556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4ae845bf-cf6c-45a4-bb8b-24c80dd95203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:08.829245 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:08.867396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 224/462] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:34:08.868473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae845bf-cf6c-45a4-bb8b-24c80dd95203 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:08.894634 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:08.914821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e826d90-f7e8-4417-af49-1e30ef210130 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:08.952258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df557e5-f895-4e2f-8694-b7cce769eea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:08.952887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df557e5-f895-4e2f-8694-b7cce769eea6 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:08.953164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df557e5-f895-4e2f-8694-b7cce769eea6 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:08.953433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df557e5-f895-4e2f-8694-b7cce769eea6 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:08.954097 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df557e5-f895-4e2f-8694-b7cce769eea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:08.954393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df557e5-f895-4e2f-8694-b7cce769eea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:08.955753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df557e5-f895-4e2f-8694-b7cce769eea6 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:08.968316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 239/463] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:08.969508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124d8d8-7b77-4295-9459-d3813eaf7361 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:08.969787 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124d8d8-7b77-4295-9459-d3813eaf7361 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:08.970136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b124d8d8-7b77-4295-9459-d3813eaf7361 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:08.975635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 240/464] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:08.976488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124d8d8-7b77-4295-9459-d3813eaf7361 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:08.983403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae845bf-cf6c-45a4-bb8b-24c80dd95203 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:09.019540 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e39fea-c4d5-44c8-84c4-ae6baeafe965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:09.020246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e39fea-c4d5-44c8-84c4-ae6baeafe965 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:09.020529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e39fea-c4d5-44c8-84c4-ae6baeafe965 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:09.020841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e39fea-c4d5-44c8-84c4-ae6baeafe965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:09.031863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e39fea-c4d5-44c8-84c4-ae6baeafe965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:09.033305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e39fea-c4d5-44c8-84c4-ae6baeafe965 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-77e39fea-c4d5-44c8-84c4-ae6baeafe965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:09.050429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:09.051521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:09.052394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:09.052394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:09.052394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:09.054132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 225/465] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:09.055386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4688d-fcb1-47d8-8bae-fcc1280238db None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:09.055877 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4688d-fcb1-47d8-8bae-fcc1280238db None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:09.056279 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4688d-fcb1-47d8-8bae-fcc1280238db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:09.076104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-01b4688d-fcb1-47d8-8bae-fcc1280238db None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:09.077676 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-01b4688d-fcb1-47d8-8bae-fcc1280238db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:09.084377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b124d8d8-7b77-4295-9459-d3813eaf7361 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:09.096645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:09.121054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f32ea-c4b8-4f15-9459-66f8714d2c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:09.121713 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f32ea-c4b8-4f15-9459-66f8714d2c49 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:09.122136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f32ea-c4b8-4f15-9459-66f8714d2c49 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:09.122539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743f32ea-c4b8-4f15-9459-66f8714d2c49 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:09.123502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f32ea-c4b8-4f15-9459-66f8714d2c49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:09.124680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743f32ea-c4b8-4f15-9459-66f8714d2c49 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:09.137268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:09.151207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 226/466] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:09.152254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4688d-fcb1-47d8-8bae-fcc1280238db None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:09.234296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 241/467] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:34:08 2022] POST /identity/v3/users => generated 246 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:34:09.236492 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f32ea-c4b8-4f15-9459-66f8714d2c49 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:09.283772 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4688d-fcb1-47d8-8bae-fcc1280238db None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:09.359846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f474867-4660-42ab-b214-fe00451c3fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:09.360709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f474867-4660-42ab-b214-fe00451c3fca None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:09.361123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f474867-4660-42ab-b214-fe00451c3fca None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:09.361557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f474867-4660-42ab-b214-fe00451c3fca None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:09.362383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f474867-4660-42ab-b214-fe00451c3fca None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:09.363540 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f474867-4660-42ab-b214-fe00451c3fca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:09.365303 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f474867-4660-42ab-b214-fe00451c3fca None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:09.394047 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 227/468] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:09.399254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f474867-4660-42ab-b214-fe00451c3fca None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:09.418222 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:09.450041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743f32ea-c4b8-4f15-9459-66f8714d2c49 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:09.486466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0eb43-9f69-4790-852a-a0d61c67d075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:09.487020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0eb43-9f69-4790-852a-a0d61c67d075 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:09.487293 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0eb43-9f69-4790-852a-a0d61c67d075 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:09.487561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0eb43-9f69-4790-852a-a0d61c67d075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:09.502073 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a0eb43-9f69-4790-852a-a0d61c67d075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:09.503519 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a0eb43-9f69-4790-852a-a0d61c67d075 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:09.513787 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f474867-4660-42ab-b214-fe00451c3fca None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-86a0eb43-9f69-4790-852a-a0d61c67d075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:09.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:09.529103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:09.530009 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:09.530009 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:09.530009 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:09.530009 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 242/469] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:09.530009 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0eb43-9f69-4790-852a-a0d61c67d075 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:09.575014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:09.593916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab52ac7-5a94-4d05-bd32-1e57c6fd28d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:09.594530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab52ac7-5a94-4d05-bd32-1e57c6fd28d9 None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:09.594804 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab52ac7-5a94-4d05-bd32-1e57c6fd28d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:09.595075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab52ac7-5a94-4d05-bd32-1e57c6fd28d9 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:09.595912 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab52ac7-5a94-4d05-bd32-1e57c6fd28d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:09.597913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab52ac7-5a94-4d05-bd32-1e57c6fd28d9 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:09.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0eb43-9f69-4790-852a-a0d61c67d075 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:09.668786 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaf3d8-a2c2-44fc-a604-3da26ba92bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:09.669442 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaf3d8-a2c2-44fc-a604-3da26ba92bd6 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:09.669718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaf3d8-a2c2-44fc-a604-3da26ba92bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:09.670015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afaf3d8-a2c2-44fc-a604-3da26ba92bd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:09.680938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afaf3d8-a2c2-44fc-a604-3da26ba92bd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:09.682546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afaf3d8-a2c2-44fc-a604-3da26ba92bd6 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:09.687777 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 228/470] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:34:09 2022] POST /identity/v3/users => generated 248 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:34:09.695276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab52ac7-5a94-4d05-bd32-1e57c6fd28d9 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-1afaf3d8-a2c2-44fc-a604-3da26ba92bd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:09.705191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:09.706749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:09.707680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:09.707680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:09.707680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:09.707842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 243/471] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:09.713828 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaf3d8-a2c2-44fc-a604-3da26ba92bd6 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:09.826381 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:09.852152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab52ac7-5a94-4d05-bd32-1e57c6fd28d9 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:09.908402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b118acab-d780-48b9-a0fb-680822df50c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:09.909127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118acab-d780-48b9-a0fb-680822df50c7 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:09.909406 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118acab-d780-48b9-a0fb-680822df50c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:09.909676 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b118acab-d780-48b9-a0fb-680822df50c7 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:09.910358 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118acab-d780-48b9-a0fb-680822df50c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:09.910695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118acab-d780-48b9-a0fb-680822df50c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:09.912622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b118acab-d780-48b9-a0fb-680822df50c7 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:09.929490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 229/472] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:09.929490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24c58b-8fb0-451a-838d-3bf1b153fcbe None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:09.929490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24c58b-8fb0-451a-838d-3bf1b153fcbe None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:09.929490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d24c58b-8fb0-451a-838d-3bf1b153fcbe None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:09.929490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 230/473] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:09.934302 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24c58b-8fb0-451a-838d-3bf1b153fcbe None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:09.941280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:09.943938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afaf3d8-a2c2-44fc-a604-3da26ba92bd6 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:09.992958 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b10f90b-f743-4d4f-8503-b83632df0002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:09.993838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10f90b-f743-4d4f-8503-b83632df0002 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:09.994368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10f90b-f743-4d4f-8503-b83632df0002 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:09.994791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b10f90b-f743-4d4f-8503-b83632df0002 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:09.995571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10f90b-f743-4d4f-8503-b83632df0002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:09.996012 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10f90b-f743-4d4f-8503-b83632df0002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:09.998229 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b10f90b-f743-4d4f-8503-b83632df0002 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:10.011595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 244/474] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:10.013153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831b88c-8432-4940-8272-4c5c01bd2807 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:10.013576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831b88c-8432-4940-8272-4c5c01bd2807 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:10.014069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f831b88c-8432-4940-8272-4c5c01bd2807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:10.033660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f831b88c-8432-4940-8272-4c5c01bd2807 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:10.035141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f831b88c-8432-4940-8272-4c5c01bd2807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:10.058545 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d24c58b-8fb0-451a-838d-3bf1b153fcbe None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:10.091568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:10.098066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf10bc-a4e0-44df-a80a-7592ed132d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:10.098814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf10bc-a4e0-44df-a80a-7592ed132d5d None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:10.110112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf10bc-a4e0-44df-a80a-7592ed132d5d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:10.110112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf10bc-a4e0-44df-a80a-7592ed132d5d None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:10.110112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf10bc-a4e0-44df-a80a-7592ed132d5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:10.110112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf10bc-a4e0-44df-a80a-7592ed132d5d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:10.110392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 245/475] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:10.111159 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831b88c-8432-4940-8272-4c5c01bd2807 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:10.203728 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:10.213414 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f831b88c-8432-4940-8272-4c5c01bd2807 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:10.222282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 231/476] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:34:09 2022] POST /identity/v3/users => generated 249 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:34:10.266735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e5996-2adf-473b-80f9-7ba6dce709e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:10.267625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e5996-2adf-473b-80f9-7ba6dce709e2 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:10.268085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e5996-2adf-473b-80f9-7ba6dce709e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:10.268739 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e5996-2adf-473b-80f9-7ba6dce709e2 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:10.269397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e5996-2adf-473b-80f9-7ba6dce709e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:10.270949 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e5996-2adf-473b-80f9-7ba6dce709e2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:10.382307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 246/477] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:34:10 2022] POST /identity/v3/users => generated 248 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:34:10.415401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf10bc-a4e0-44df-a80a-7592ed132d5d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:10.539538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf10bc-a4e0-44df-a80a-7592ed132d5d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:10.578935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fe7c7-32de-46a5-aa48-ae93e37650c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:10.579543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fe7c7-32de-46a5-aa48-ae93e37650c5 None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:10.579864 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fe7c7-32de-46a5-aa48-ae93e37650c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:10.580137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fe7c7-32de-46a5-aa48-ae93e37650c5 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:10.580812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fe7c7-32de-46a5-aa48-ae93e37650c5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:10.582267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fe7c7-32de-46a5-aa48-ae93e37650c5 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:10.616592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.sql.core [None req-768fe7c7-32de-46a5-aa48-ae93e37650c5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 19:34:10.616592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 14 19:34:10.616592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [parameters: {'id': 'c9340025d649455588141e3a0018a6b7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 14 19:34:10.616592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93426) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-768fe7c7-32de-46a5-aa48-ae93e37650c5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application conn.query(q) Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application result.read() Sep 14 19:34:10.633884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return ref.to_dict() Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 19:34:10.634763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application next(self.gen) Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application yield resource Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application next(self.gen) Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise self.value Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application session.commit() Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 19:34:10.635686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.session.flush() Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application compat.raise_( Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 19:34:10.636566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 19:34:10.637466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application conn.query(q) Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application result.read() Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 19:34:10.639404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [parameters: {'id': 'c9340025d649455588141e3a0018a6b7', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:10.640910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 14 19:34:10.642519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:10.645071 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 232/478] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Sep 14 19:34:10 2022] POST /identity/v3/roles => generated 144 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:10.645204 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e5996-2adf-473b-80f9-7ba6dce709e2 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:10.755861 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e5996-2adf-473b-80f9-7ba6dce709e2 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:10.795397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242435ec-3741-41bf-98af-504a259bf8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:10.796269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242435ec-3741-41bf-98af-504a259bf8c0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:10.796682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242435ec-3741-41bf-98af-504a259bf8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:10.797089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242435ec-3741-41bf-98af-504a259bf8c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:10.816707 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242435ec-3741-41bf-98af-504a259bf8c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-242435ec-3741-41bf-98af-504a259bf8c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:10.822371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:10.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:10.824387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:10.824387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:10.824387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:10.824387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:10.824387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:10.824387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:10.824387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:10.824387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:10.824387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:10.824387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:10.824387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 19:34:10.824387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:10.826659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 247/479] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:34:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:10.832247 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fe7c7-32de-46a5-aa48-ae93e37650c5 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:10.907751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:10.959172 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fe7c7-32de-46a5-aa48-ae93e37650c5 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:10.999003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93420eef-bff8-44b5-ad50-b72da8a6f542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:10.999620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93420eef-bff8-44b5-ad50-b72da8a6f542 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:10.999901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93420eef-bff8-44b5-ad50-b72da8a6f542 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:11.000178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93420eef-bff8-44b5-ad50-b72da8a6f542 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:11.000834 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93420eef-bff8-44b5-ad50-b72da8a6f542 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:11.001204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93420eef-bff8-44b5-ad50-b72da8a6f542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:11.002627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93420eef-bff8-44b5-ad50-b72da8a6f542 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:11.011353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 233/480] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:11.943464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c20519-25c6-4f41-9a27-9d0eb711397a None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:11.943965 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c20519-25c6-4f41-9a27-9d0eb711397a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:11.944040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c20519-25c6-4f41-9a27-9d0eb711397a None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:11.945595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 248/481] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:11.951588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d74ec-75f6-43ba-936e-cc19f6b32cec None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:11.951983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d74ec-75f6-43ba-936e-cc19f6b32cec None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:11.952164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d74ec-75f6-43ba-936e-cc19f6b32cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:11.983015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-4e1d74ec-75f6-43ba-936e-cc19f6b32cec None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:11.984207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4e1d74ec-75f6-43ba-936e-cc19f6b32cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:12.069914 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 234/482] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:34:12.430379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c20519-25c6-4f41-9a27-9d0eb711397a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:12.440719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:12.562260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c20519-25c6-4f41-9a27-9d0eb711397a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:12.603837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa1ffce-9a6d-45e7-a6eb-968085b2b3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:12.604465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1ffce-9a6d-45e7-a6eb-968085b2b3d3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:12.604736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1ffce-9a6d-45e7-a6eb-968085b2b3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:12.605002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1ffce-9a6d-45e7-a6eb-968085b2b3d3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:12.626709 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa1ffce-9a6d-45e7-a6eb-968085b2b3d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-aaa1ffce-9a6d-45e7-a6eb-968085b2b3d3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:12.631886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:12.632793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:12.633612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:12.633612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:12.633612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:12.633612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:12.633612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:12.633612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:12.633612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:12.633612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:12.633612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:12.633612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:12.633612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:12.633612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:12.633612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 249/483] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:12.639440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d74ec-75f6-43ba-936e-cc19f6b32cec None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:12.665140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d969204-a8aa-476c-858a-177fa9a64ac4 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:12.665428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d969204-a8aa-476c-858a-177fa9a64ac4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:12.665682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d969204-a8aa-476c-858a-177fa9a64ac4 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:12.667490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 250/484] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:34:12.673115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede707c-7468-4b38-b3ed-4566bc23230f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:12.673388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede707c-7468-4b38-b3ed-4566bc23230f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:12.673655 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede707c-7468-4b38-b3ed-4566bc23230f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:12.692232 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-fede707c-7468-4b38-b3ed-4566bc23230f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:12.693603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fede707c-7468-4b38-b3ed-4566bc23230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:12.714173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:12.802661 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:12.817554 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d74ec-75f6-43ba-936e-cc19f6b32cec None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:12.819171 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 251/485] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:34:12.857474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a157d-6729-4f3b-9d59-f83db528c4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:12.857474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a157d-6729-4f3b-9d59-f83db528c4d2 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:12.857474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a157d-6729-4f3b-9d59-f83db528c4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:12.857474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a157d-6729-4f3b-9d59-f83db528c4d2 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:12.858102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a157d-6729-4f3b-9d59-f83db528c4d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:12.858102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a157d-6729-4f3b-9d59-f83db528c4d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:12.860347 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a157d-6729-4f3b-9d59-f83db528c4d2 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:12.868071 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 235/486] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:12.876241 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede707c-7468-4b38-b3ed-4566bc23230f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:12.985506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede707c-7468-4b38-b3ed-4566bc23230f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:13.025216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cbad0e-9210-4645-b824-0562fd763857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:13.026099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbad0e-9210-4645-b824-0562fd763857 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:13.026521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbad0e-9210-4645-b824-0562fd763857 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:13.026933 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cbad0e-9210-4645-b824-0562fd763857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:13.041653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cbad0e-9210-4645-b824-0562fd763857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:13.043294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cbad0e-9210-4645-b824-0562fd763857 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-42cbad0e-9210-4645-b824-0562fd763857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:13.064977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:13.070221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:13.071286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:13.071286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:13.071286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:13.072971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 252/487] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:13.078728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a157d-6729-4f3b-9d59-f83db528c4d2 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:13.146119 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cbad0e-9210-4645-b824-0562fd763857 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:13.228107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a157d-6729-4f3b-9d59-f83db528c4d2 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:13.232702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:13.232822 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:13.265091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4601167-295e-4686-9f82-e73afc946c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:13.265624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4601167-295e-4686-9f82-e73afc946c9d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:13.265926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4601167-295e-4686-9f82-e73afc946c9d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:13.266199 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4601167-295e-4686-9f82-e73afc946c9d None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:13.266840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4601167-295e-4686-9f82-e73afc946c9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:13.267139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4601167-295e-4686-9f82-e73afc946c9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:13.268684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4601167-295e-4686-9f82-e73afc946c9d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:13.283162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 236/488] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:13.289051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4601167-295e-4686-9f82-e73afc946c9d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:13.343530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cbad0e-9210-4645-b824-0562fd763857 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:13.378128 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd24e0-5cdf-4ffd-9947-d89cb4a71b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:13.378947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd24e0-5cdf-4ffd-9947-d89cb4a71b40 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:13.379446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd24e0-5cdf-4ffd-9947-d89cb4a71b40 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:13.379954 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd24e0-5cdf-4ffd-9947-d89cb4a71b40 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:13.390610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd24e0-5cdf-4ffd-9947-d89cb4a71b40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-b6fd24e0-5cdf-4ffd-9947-d89cb4a71b40 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:13.395826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:13.396699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:13.397527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:13.397527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:13.397527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:13.397527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:13.397527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:13.397527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:13.397527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:13.397527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:13.397527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:13.397527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:13.397527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:13.397527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:13.397527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 253/489] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:13.398795 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b9968-0f60-4036-97d8-baed587bdcc5 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:13.398795 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b9968-0f60-4036-97d8-baed587bdcc5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:13.398795 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901b9968-0f60-4036-97d8-baed587bdcc5 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:13.400064 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 254/490] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:13.400989 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced8b3d-8171-4906-99e2-81516fc33820 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:13.401246 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced8b3d-8171-4906-99e2-81516fc33820 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:13.401665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced8b3d-8171-4906-99e2-81516fc33820 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:13.406986 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 255/491] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:13.406986 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced8b3d-8171-4906-99e2-81516fc33820 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:13.424092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4601167-295e-4686-9f82-e73afc946c9d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:13.459500 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e1473-ce24-435c-8b06-13a3d8905e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:13.460125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e1473-ce24-435c-8b06-13a3d8905e48 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:13.460400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e1473-ce24-435c-8b06-13a3d8905e48 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:13.460663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e1473-ce24-435c-8b06-13a3d8905e48 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:13.467966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e1473-ce24-435c-8b06-13a3d8905e48 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:13.469408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e1473-ce24-435c-8b06-13a3d8905e48 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-3e9e1473-ce24-435c-8b06-13a3d8905e48 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:13.482862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:13.483812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:13.494413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:13.494413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:13.494413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:13.494413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:13.494413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:13.494413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:13.494413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:13.494413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:13.494413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:34:13.494413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:13.494413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 237/492] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 19:34:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:13.494413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9b0a5-1113-4ebb-9c10-253bc52d7c08 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:13.494413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9b0a5-1113-4ebb-9c10-253bc52d7c08 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:13.495088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9b0a5-1113-4ebb-9c10-253bc52d7c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:13.507286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e8c9b0a5-1113-4ebb-9c10-253bc52d7c08 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:13.508565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e8c9b0a5-1113-4ebb-9c10-253bc52d7c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:13.517055 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced8b3d-8171-4906-99e2-81516fc33820 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:13.526388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:13.529073 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:13.556320 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15b9eeb-5854-4ac7-a9eb-bacf1a364960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:13.557333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9eeb-5854-4ac7-a9eb-bacf1a364960 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:13.557763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9eeb-5854-4ac7-a9eb-bacf1a364960 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:13.558344 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15b9eeb-5854-4ac7-a9eb-bacf1a364960 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:13.559533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b9eeb-5854-4ac7-a9eb-bacf1a364960 None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:13.560802 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b9eeb-5854-4ac7-a9eb-bacf1a364960 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:13.562896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15b9eeb-5854-4ac7-a9eb-bacf1a364960 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:13.571767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 256/493] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:13.577891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a6873-de76-4e44-b935-90a770c1bc21 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:13.578472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a6873-de76-4e44-b935-90a770c1bc21 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:13.579089 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 238/494] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:13.579428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a6873-de76-4e44-b935-90a770c1bc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:13.579967 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9b0a5-1113-4ebb-9c10-253bc52d7c08 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:13.611443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-991a6873-de76-4e44-b935-90a770c1bc21 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:13.613211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-991a6873-de76-4e44-b935-90a770c1bc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:13.683306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9b0a5-1113-4ebb-9c10-253bc52d7c08 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:13.706175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 257/495] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:13.708086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a6873-de76-4e44-b935-90a770c1bc21 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:13.733662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f792e0-0919-47e2-9fa1-57f7d2fafc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:13.734324 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f792e0-0919-47e2-9fa1-57f7d2fafc19 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:13.734658 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f792e0-0919-47e2-9fa1-57f7d2fafc19 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:13.734854 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f792e0-0919-47e2-9fa1-57f7d2fafc19 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:13.735639 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f792e0-0919-47e2-9fa1-57f7d2fafc19 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:13.735984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f792e0-0919-47e2-9fa1-57f7d2fafc19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:13.737414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f792e0-0919-47e2-9fa1-57f7d2fafc19 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:13.808458 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 239/496] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 19:34:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:13.814515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f792e0-0919-47e2-9fa1-57f7d2fafc19 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:13.821305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:13.825087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a6873-de76-4e44-b935-90a770c1bc21 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:13.836115 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:13.875366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46603eb9-2fde-43bf-b10b-787afb429c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:13.876014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46603eb9-2fde-43bf-b10b-787afb429c1a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:13.876291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46603eb9-2fde-43bf-b10b-787afb429c1a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:13.876563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46603eb9-2fde-43bf-b10b-787afb429c1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:13.892487 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46603eb9-2fde-43bf-b10b-787afb429c1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:13.895091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46603eb9-2fde-43bf-b10b-787afb429c1a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-46603eb9-2fde-43bf-b10b-787afb429c1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:13.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:13.912664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:13.913533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:13.913533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:13.913533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:13.913533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 258/497] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:13.913759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46603eb9-2fde-43bf-b10b-787afb429c1a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:13.953114 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f792e0-0919-47e2-9fa1-57f7d2fafc19 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:13.987116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e33630-b7ad-4014-b177-f0dfd16512c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:13.987714 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e33630-b7ad-4014-b177-f0dfd16512c3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:13.987981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e33630-b7ad-4014-b177-f0dfd16512c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:13.988243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e33630-b7ad-4014-b177-f0dfd16512c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:14.018731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46603eb9-2fde-43bf-b10b-787afb429c1a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:14.053652 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e37dfc-2f83-4878-8cbd-01f1bfec8981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:14.054307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e37dfc-2f83-4878-8cbd-01f1bfec8981 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:14.054589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e37dfc-2f83-4878-8cbd-01f1bfec8981 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:14.054863 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e37dfc-2f83-4878-8cbd-01f1bfec8981 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:14.058150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e33630-b7ad-4014-b177-f0dfd16512c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:14.059823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e33630-b7ad-4014-b177-f0dfd16512c3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:14.065170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e37dfc-2f83-4878-8cbd-01f1bfec8981 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-47e37dfc-2f83-4878-8cbd-01f1bfec8981 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:14.070633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:14.071518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:14.072323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:14.072323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:14.072323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:14.072323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:14.072323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:14.072323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:14.072323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:14.072323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:14.072323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:14.072323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:14.072323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:14.072323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:14.072323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 259/498] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:14.072993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0ba71-b959-47fc-8005-00d37df9fd24 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:14.073235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0ba71-b959-47fc-8005-00d37df9fd24 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:14.073427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0ba71-b959-47fc-8005-00d37df9fd24 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:14.074860 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 260/499] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:14.075972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0ba71-b959-47fc-8005-00d37df9fd24 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:14.077470 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:14.092071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:14.163458 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 240/500] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 19:34:14.164825 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e33630-b7ad-4014-b177-f0dfd16512c3 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:14.178987 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0ba71-b959-47fc-8005-00d37df9fd24 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:14.212617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baa8276-65ee-48a8-b224-c7277558ff8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:14.213207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa8276-65ee-48a8-b224-c7277558ff8d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:14.213479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa8276-65ee-48a8-b224-c7277558ff8d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:14.213746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baa8276-65ee-48a8-b224-c7277558ff8d None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:14.214418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa8276-65ee-48a8-b224-c7277558ff8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:14.214714 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa8276-65ee-48a8-b224-c7277558ff8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:14.216130 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baa8276-65ee-48a8-b224-c7277558ff8d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:14.227362 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 261/501] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:14.228493 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86921e26-992d-45be-9eea-90d7a7c1ae95 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:14.228769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86921e26-992d-45be-9eea-90d7a7c1ae95 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:14.229036 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86921e26-992d-45be-9eea-90d7a7c1ae95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:14.248565 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-86921e26-992d-45be-9eea-90d7a7c1ae95 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:14.249774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-86921e26-992d-45be-9eea-90d7a7c1ae95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:14.264523 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e33630-b7ad-4014-b177-f0dfd16512c3 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:14.277376 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:14.298091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd9a72-ef87-4c09-995d-d283903f91a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:14.298740 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd9a72-ef87-4c09-995d-d283903f91a4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:14.299020 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd9a72-ef87-4c09-995d-d283903f91a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:14.299288 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cd9a72-ef87-4c09-995d-d283903f91a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:14.309121 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cd9a72-ef87-4c09-995d-d283903f91a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-48cd9a72-ef87-4c09-995d-d283903f91a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:14.315042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:14.315909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:14.316701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:14.316701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:14.316701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:14.316701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:14.316701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:14.316701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:14.316701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:14.316701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:14.316701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:14.316701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:14.316701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:14.316701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:14.317642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 262/502] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:14.317642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86921e26-992d-45be-9eea-90d7a7c1ae95 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:14.317968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 241/503] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:14.318879 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd9a72-ef87-4c09-995d-d283903f91a4 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:14.330516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:14.420728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cd9a72-ef87-4c09-995d-d283903f91a4 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:14.427937 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86921e26-992d-45be-9eea-90d7a7c1ae95 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:14.461611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a27f10-18ee-42e0-b25d-49621d729eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:14.462393 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a27f10-18ee-42e0-b25d-49621d729eb9 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:14.462736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a27f10-18ee-42e0-b25d-49621d729eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:14.463062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a27f10-18ee-42e0-b25d-49621d729eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:14.464862 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df70d8-df39-4fd5-8451-b8e8b8c105f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:14.465457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df70d8-df39-4fd5-8451-b8e8b8c105f4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:14.465744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df70d8-df39-4fd5-8451-b8e8b8c105f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:14.466625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df70d8-df39-4fd5-8451-b8e8b8c105f4 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:14.467282 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df70d8-df39-4fd5-8451-b8e8b8c105f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:14.467583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df70d8-df39-4fd5-8451-b8e8b8c105f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:14.468942 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df70d8-df39-4fd5-8451-b8e8b8c105f4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:14.473748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a27f10-18ee-42e0-b25d-49621d729eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:14.475201 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a27f10-18ee-42e0-b25d-49621d729eb9 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:14.482323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 263/504] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:14.482323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df70d8-df39-4fd5-8451-b8e8b8c105f4 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-b5a27f10-18ee-42e0-b25d-49621d729eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:14.490588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:14.491472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:14.492308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:14.492308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:14.492308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:14.492308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 242/505] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:14.492529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a27f10-18ee-42e0-b25d-49621d729eb9 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:14.518795 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:14.553581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:14.591809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df70d8-df39-4fd5-8451-b8e8b8c105f4 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:14.594969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a27f10-18ee-42e0-b25d-49621d729eb9 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:14.628925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5652c-8f91-4f5d-9fea-082550e3958b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:14.629530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5652c-8f91-4f5d-9fea-082550e3958b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:14.629821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5652c-8f91-4f5d-9fea-082550e3958b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:14.630114 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5652c-8f91-4f5d-9fea-082550e3958b None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:14.630769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5652c-8f91-4f5d-9fea-082550e3958b None admin] RBAC: Adding query filter params (name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:14.631074 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5652c-8f91-4f5d-9fea-082550e3958b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:14.632206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2cd69-3389-4af1-b77d-10c2e9f1a7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:14.632455 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5652c-8f91-4f5d-9fea-082550e3958b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:14.632829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2cd69-3389-4af1-b77d-10c2e9f1a7ac None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:14.633106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2cd69-3389-4af1-b77d-10c2e9f1a7ac None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:14.633377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb2cd69-3389-4af1-b77d-10c2e9f1a7ac None admin] PATH_INFO: `/v3/users/glance` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:14.640844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 243/506] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:14.641629 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5652c-8f91-4f5d-9fea-082550e3958b None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:14.642665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2cd69-3389-4af1-b77d-10c2e9f1a7ac None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:14.644087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb2cd69-3389-4af1-b77d-10c2e9f1a7ac None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-7cb2cd69-3389-4af1-b77d-10c2e9f1a7ac None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:14.656911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:14.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:14.658775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:14.658775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:14.658775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:14.658775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:14.658775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:14.658775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:14.658775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:14.658775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:14.658775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 19:34:14.658775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:14.658775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 264/507] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:14.658775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2cd69-3389-4af1-b77d-10c2e9f1a7ac None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:14.728289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:14.742403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5652c-8f91-4f5d-9fea-082550e3958b None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:14.758153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb2cd69-3389-4af1-b77d-10c2e9f1a7ac None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:14.775952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67391659-cffe-42ba-8854-c1c22fa83622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:14.776577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67391659-cffe-42ba-8854-c1c22fa83622 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:14.776850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67391659-cffe-42ba-8854-c1c22fa83622 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:14.777122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67391659-cffe-42ba-8854-c1c22fa83622 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:14.780888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:14.787390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67391659-cffe-42ba-8854-c1c22fa83622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:14.789075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67391659-cffe-42ba-8854-c1c22fa83622 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:14.798222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc30ff-bd25-476d-a887-8b35cbe9e04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:14.798222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc30ff-bd25-476d-a887-8b35cbe9e04e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:14.798222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc30ff-bd25-476d-a887-8b35cbe9e04e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:14.798222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc30ff-bd25-476d-a887-8b35cbe9e04e None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:14.798570 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc30ff-bd25-476d-a887-8b35cbe9e04e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:14.798570 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc30ff-bd25-476d-a887-8b35cbe9e04e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:14.798570 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc30ff-bd25-476d-a887-8b35cbe9e04e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-67391659-cffe-42ba-8854-c1c22fa83622 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:14.803364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:14.804254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:14.805107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:14.805107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:14.805107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:14.805107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 244/508] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:14.805331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67391659-cffe-42ba-8854-c1c22fa83622 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:14.806138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 265/509] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:14.806836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc30ff-bd25-476d-a887-8b35cbe9e04e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:14.908707 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67391659-cffe-42ba-8854-c1c22fa83622 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:14.910646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc30ff-bd25-476d-a887-8b35cbe9e04e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:14.943105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563dc2f-1abe-404f-b4b9-08cdbb7b556d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:14.943720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563dc2f-1abe-404f-b4b9-08cdbb7b556d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:14.943992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563dc2f-1abe-404f-b4b9-08cdbb7b556d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:14.944261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563dc2f-1abe-404f-b4b9-08cdbb7b556d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:14.948976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898648c4-2fb8-4dea-bcbd-2fd148abe069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:14.949594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898648c4-2fb8-4dea-bcbd-2fd148abe069 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:14.949904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898648c4-2fb8-4dea-bcbd-2fd148abe069 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:14.950177 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898648c4-2fb8-4dea-bcbd-2fd148abe069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:14.951087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:14.954748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563dc2f-1abe-404f-b4b9-08cdbb7b556d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:14.956181 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b563dc2f-1abe-404f-b4b9-08cdbb7b556d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:14.960106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898648c4-2fb8-4dea-bcbd-2fd148abe069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:14.961527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898648c4-2fb8-4dea-bcbd-2fd148abe069 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-b563dc2f-1abe-404f-b4b9-08cdbb7b556d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:14.970159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:14.971182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:14.972040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:14.972040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:14.972040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:14.972040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 245/510] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:14.972040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563dc2f-1abe-404f-b4b9-08cdbb7b556d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-898648c4-2fb8-4dea-bcbd-2fd148abe069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:14.976374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:14.977424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:14.978482 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:14.978482 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:14.978482 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:14.978482 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 266/511] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:14.978482 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898648c4-2fb8-4dea-bcbd-2fd148abe069 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:15.014412 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:15.070795 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563dc2f-1abe-404f-b4b9-08cdbb7b556d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:15.076882 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898648c4-2fb8-4dea-bcbd-2fd148abe069 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:15.102857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a21e2d-4a22-4d26-80b0-0fa126612355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:15.103450 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a21e2d-4a22-4d26-80b0-0fa126612355 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:15.103725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a21e2d-4a22-4d26-80b0-0fa126612355 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:15.103993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a21e2d-4a22-4d26-80b0-0fa126612355 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:15.104698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a21e2d-4a22-4d26-80b0-0fa126612355 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:15.104998 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a21e2d-4a22-4d26-80b0-0fa126612355 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:15.106984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a21e2d-4a22-4d26-80b0-0fa126612355 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:15.110313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f3ec3-6fb7-4e99-83d3-8b28fd99a0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:15.110906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f3ec3-6fb7-4e99-83d3-8b28fd99a0ca None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:15.111185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f3ec3-6fb7-4e99-83d3-8b28fd99a0ca None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:15.111457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235f3ec3-6fb7-4e99-83d3-8b28fd99a0ca None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:15.112078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f3ec3-6fb7-4e99-83d3-8b28fd99a0ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:15.112378 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f3ec3-6fb7-4e99-83d3-8b28fd99a0ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:15.113730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235f3ec3-6fb7-4e99-83d3-8b28fd99a0ca None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:15.123449 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 246/512] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Sep 14 19:34:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:15.124321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a21e2d-4a22-4d26-80b0-0fa126612355 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:15.125551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 267/513] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:15.126384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f3ec3-6fb7-4e99-83d3-8b28fd99a0ca None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:15.178166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:15.218454 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:15.224567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235f3ec3-6fb7-4e99-83d3-8b28fd99a0ca None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:15.227919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a21e2d-4a22-4d26-80b0-0fa126612355 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:15.257357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95440584-81d2-4a66-b002-63bc42f7baea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:15.258017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95440584-81d2-4a66-b002-63bc42f7baea None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:15.258286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95440584-81d2-4a66-b002-63bc42f7baea None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:15.258552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95440584-81d2-4a66-b002-63bc42f7baea None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:15.259172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95440584-81d2-4a66-b002-63bc42f7baea None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:15.259466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95440584-81d2-4a66-b002-63bc42f7baea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:15.260821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95440584-81d2-4a66-b002-63bc42f7baea None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:15.261699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f695c4-8830-437f-9b3f-b51406fe82b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:15.262343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f695c4-8830-437f-9b3f-b51406fe82b0 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:15.262616 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f695c4-8830-437f-9b3f-b51406fe82b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:15.263801 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f695c4-8830-437f-9b3f-b51406fe82b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:15.270293 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f695c4-8830-437f-9b3f-b51406fe82b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:15.271717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f695c4-8830-437f-9b3f-b51406fe82b0 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:15.272335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 268/514] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:15.273211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95440584-81d2-4a66-b002-63bc42f7baea None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-53f695c4-8830-437f-9b3f-b51406fe82b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:15.283922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:15.284794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:15.285672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:15.285672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:15.285672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:15.285672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:15.285672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:15.285672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:15.285672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:15.285672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:15.286120 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 247/515] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:15.286828 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f695c4-8830-437f-9b3f-b51406fe82b0 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:15.376685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95440584-81d2-4a66-b002-63bc42f7baea None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:15.383563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f695c4-8830-437f-9b3f-b51406fe82b0 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:15.393280 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:15.409116 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6508c-5f9f-45ae-8fbe-7103402990d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:15.409712 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6508c-5f9f-45ae-8fbe-7103402990d6 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:15.410011 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6508c-5f9f-45ae-8fbe-7103402990d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:15.410280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6508c-5f9f-45ae-8fbe-7103402990d6 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:15.410945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6508c-5f9f-45ae-8fbe-7103402990d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:15.411242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6508c-5f9f-45ae-8fbe-7103402990d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:15.412590 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d6508c-5f9f-45ae-8fbe-7103402990d6 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:15.418266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41e4ac-f403-4078-8547-d00a99f69add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:15.418871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e4ac-f403-4078-8547-d00a99f69add None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:15.419192 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e4ac-f403-4078-8547-d00a99f69add None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:15.419463 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be41e4ac-f403-4078-8547-d00a99f69add None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:15.425164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 269/516] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:15.425991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6508c-5f9f-45ae-8fbe-7103402990d6 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:15.445409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:15.486080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41e4ac-f403-4078-8547-d00a99f69add None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:15.487537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be41e4ac-f403-4078-8547-d00a99f69add None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:15.525136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d6508c-5f9f-45ae-8fbe-7103402990d6 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:15.559169 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790ded6-ae34-4387-94cc-f0db27edc73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:15.560315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790ded6-ae34-4387-94cc-f0db27edc73b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:15.560588 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790ded6-ae34-4387-94cc-f0db27edc73b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:15.560863 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790ded6-ae34-4387-94cc-f0db27edc73b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:15.567921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790ded6-ae34-4387-94cc-f0db27edc73b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:15.569357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4790ded6-ae34-4387-94cc-f0db27edc73b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-4790ded6-ae34-4387-94cc-f0db27edc73b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:15.580602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:15.581505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:15.582475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:15.582475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:15.582475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:15.582475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:15.582475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:15.582475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:15.582475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:15.582475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:15.582475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 19:34:15.582475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:15.582475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 270/517] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:15.582475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790ded6-ae34-4387-94cc-f0db27edc73b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:15.587914 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 248/518] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:34:15.587914 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41e4ac-f403-4078-8547-d00a99f69add None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:15.595742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:15.660230 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:15.682439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790ded6-ae34-4387-94cc-f0db27edc73b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:15.686483 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be41e4ac-f403-4078-8547-d00a99f69add None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:15.715237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881cde37-c086-4cc6-836a-76025cbdfb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:15.715829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cde37-c086-4cc6-836a-76025cbdfb69 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:15.716105 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cde37-c086-4cc6-836a-76025cbdfb69 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:15.716371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cde37-c086-4cc6-836a-76025cbdfb69 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:15.717026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881cde37-c086-4cc6-836a-76025cbdfb69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:15.718444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881cde37-c086-4cc6-836a-76025cbdfb69 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:15.718846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9295e-4ac0-484f-946f-9d5730e7efaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:15.719444 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9295e-4ac0-484f-946f-9d5730e7efaa None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:15.719722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9295e-4ac0-484f-946f-9d5730e7efaa None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:15.719992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9295e-4ac0-484f-946f-9d5730e7efaa None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:15.720666 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9295e-4ac0-484f-946f-9d5730e7efaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:15.720970 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9295e-4ac0-484f-946f-9d5730e7efaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:15.722414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9295e-4ac0-484f-946f-9d5730e7efaa None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:15.739810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 249/519] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:15.742801 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9295e-4ac0-484f-946f-9d5730e7efaa None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:15.801838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 271/520] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:34:15 2022] POST /identity/v3/users => generated 247 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:34:15.801838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881cde37-c086-4cc6-836a-76025cbdfb69 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:15.806243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:15.841386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9295e-4ac0-484f-946f-9d5730e7efaa None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:15.889365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f88ba5-b73a-4ecf-a7c7-796833ca1e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:15.890045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f88ba5-b73a-4ecf-a7c7-796833ca1e01 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:15.890318 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f88ba5-b73a-4ecf-a7c7-796833ca1e01 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:15.890585 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f88ba5-b73a-4ecf-a7c7-796833ca1e01 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:15.898394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f88ba5-b73a-4ecf-a7c7-796833ca1e01 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:15.899995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f88ba5-b73a-4ecf-a7c7-796833ca1e01 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-a7f88ba5-b73a-4ecf-a7c7-796833ca1e01 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:15.912769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:15.913899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:15.914817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:15.914817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:15.914817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:15.914817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:15.914817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:15.914817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:15.914817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:15.914817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:15.914817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 19:34:15.914817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:15.914817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 250/521] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:15.914817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f88ba5-b73a-4ecf-a7c7-796833ca1e01 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:15.930188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881cde37-c086-4cc6-836a-76025cbdfb69 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:15.974429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:15.980684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dadc99-f48c-4d35-bb5f-56345d4157a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:15.982347 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dadc99-f48c-4d35-bb5f-56345d4157a7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:15.982347 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dadc99-f48c-4d35-bb5f-56345d4157a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:15.982586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dadc99-f48c-4d35-bb5f-56345d4157a7 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:15.983300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dadc99-f48c-4d35-bb5f-56345d4157a7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:15.989466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dadc99-f48c-4d35-bb5f-56345d4157a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:15.989466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dadc99-f48c-4d35-bb5f-56345d4157a7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:16.024841 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 272/522] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Sep 14 19:34:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:16.025694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dadc99-f48c-4d35-bb5f-56345d4157a7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:16.041109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f88ba5-b73a-4ecf-a7c7-796833ca1e01 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:16.070055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:16.078266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341bec0-35b0-4fd2-8dc8-98ce2208c58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:16.078918 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341bec0-35b0-4fd2-8dc8-98ce2208c58c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:16.079202 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341bec0-35b0-4fd2-8dc8-98ce2208c58c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:16.079475 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341bec0-35b0-4fd2-8dc8-98ce2208c58c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:16.091578 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341bec0-35b0-4fd2-8dc8-98ce2208c58c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:16.093051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a341bec0-35b0-4fd2-8dc8-98ce2208c58c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-a341bec0-35b0-4fd2-8dc8-98ce2208c58c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:16.108615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:16.109620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:16.110507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:16.110507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:16.110507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:16.113933 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 251/523] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:16.115077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341bec0-35b0-4fd2-8dc8-98ce2208c58c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:16.135076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dadc99-f48c-4d35-bb5f-56345d4157a7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:16.174070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd76d16-4035-403f-8afc-e5fdb835531b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:16.174666 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd76d16-4035-403f-8afc-e5fdb835531b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:16.174940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd76d16-4035-403f-8afc-e5fdb835531b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:16.175303 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd76d16-4035-403f-8afc-e5fdb835531b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:16.176026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd76d16-4035-403f-8afc-e5fdb835531b None admin] RBAC: Adding query filter params (role.id=928c1b8875334543ab431e4f7e2acee5, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=2a4284b59f804dc59674d621d87634bc) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:16.176327 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd76d16-4035-403f-8afc-e5fdb835531b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:16.178029 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd76d16-4035-403f-8afc-e5fdb835531b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:16.184255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 273/524] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:16 2022] GET /identity/v3/role_assignments?user.id=2a4284b59f804dc59674d621d87634bc&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=928c1b8875334543ab431e4f7e2acee5 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:16.185562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd76d16-4035-403f-8afc-e5fdb835531b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:16.242820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a341bec0-35b0-4fd2-8dc8-98ce2208c58c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:16.257300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:16.291543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803a6bf-8832-43c3-abd9-d31b5e174edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:16.293092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803a6bf-8832-43c3-abd9-d31b5e174edf None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:16.293521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803a6bf-8832-43c3-abd9-d31b5e174edf None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:16.294032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803a6bf-8832-43c3-abd9-d31b5e174edf None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:16.294989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803a6bf-8832-43c3-abd9-d31b5e174edf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:16.295490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803a6bf-8832-43c3-abd9-d31b5e174edf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:16.297401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803a6bf-8832-43c3-abd9-d31b5e174edf None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:16.319757 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 252/525] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Sep 14 19:34:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:16.325964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803a6bf-8832-43c3-abd9-d31b5e174edf None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:16.326179 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd76d16-4035-403f-8afc-e5fdb835531b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:16.368365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5b4eb-74d9-4480-bed2-b9b3128cc210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:16.369219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5b4eb-74d9-4480-bed2-b9b3128cc210 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:16.369594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5b4eb-74d9-4480-bed2-b9b3128cc210 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:16.369990 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e5b4eb-74d9-4480-bed2-b9b3128cc210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:16.374833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:16.451336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803a6bf-8832-43c3-abd9-d31b5e174edf None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:16.452839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5b4eb-74d9-4480-bed2-b9b3128cc210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:16.454488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e5b4eb-74d9-4480-bed2-b9b3128cc210 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:16.499913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baff828-be81-4696-8916-647c0cfcdbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:16.501357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baff828-be81-4696-8916-647c0cfcdbc3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:16.501920 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baff828-be81-4696-8916-647c0cfcdbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:16.502354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baff828-be81-4696-8916-647c0cfcdbc3 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:16.503100 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baff828-be81-4696-8916-647c0cfcdbc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:16.503497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baff828-be81-4696-8916-647c0cfcdbc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:16.505707 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baff828-be81-4696-8916-647c0cfcdbc3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:16.514901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:16.528452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 253/526] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:16.528452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baff828-be81-4696-8916-647c0cfcdbc3 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:16.576297 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 274/527] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:34:16.577500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5b4eb-74d9-4480-bed2-b9b3128cc210 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:16.650002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:16.654670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baff828-be81-4696-8916-647c0cfcdbc3 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:16.690458 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb64db-d6c5-45d7-af40-e0f8f8ba38d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:16.691847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb64db-d6c5-45d7-af40-e0f8f8ba38d8 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:16.692885 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb64db-d6c5-45d7-af40-e0f8f8ba38d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:16.693322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb64db-d6c5-45d7-af40-e0f8f8ba38d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:16.695574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e5b4eb-74d9-4480-bed2-b9b3128cc210 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:16.735601 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c3147-2d66-4438-bc51-a077fa1bec85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:16.736467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c3147-2d66-4438-bc51-a077fa1bec85 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:16.736907 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c3147-2d66-4438-bc51-a077fa1bec85 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:16.737316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c3147-2d66-4438-bc51-a077fa1bec85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:16.744700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c3147-2d66-4438-bc51-a077fa1bec85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:16.746375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c3147-2d66-4438-bc51-a077fa1bec85 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-009c3147-2d66-4438-bc51-a077fa1bec85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:16.759083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:16.760110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:16.761151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:16.761151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:16.761151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:16.761151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:16.761151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:16.761151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:16.761151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:16.761151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:16.768187 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 275/528] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:16.769192 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c3147-2d66-4438-bc51-a077fa1bec85 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:16.773982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb64db-d6c5-45d7-af40-e0f8f8ba38d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:16.775586 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb64db-d6c5-45d7-af40-e0f8f8ba38d8 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:16.785510 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:16.882218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c3147-2d66-4438-bc51-a077fa1bec85 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:16.890798 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:16.896308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 254/529] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:34:16.897271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb64db-d6c5-45d7-af40-e0f8f8ba38d8 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:16.915311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e353d-97dc-4b37-80ed-c459c468f6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:16.915946 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e353d-97dc-4b37-80ed-c459c468f6b5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:16.916226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e353d-97dc-4b37-80ed-c459c468f6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:16.916498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e353d-97dc-4b37-80ed-c459c468f6b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:16.927615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e353d-97dc-4b37-80ed-c459c468f6b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:16.929066 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3e353d-97dc-4b37-80ed-c459c468f6b5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-ea3e353d-97dc-4b37-80ed-c459c468f6b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:16.944826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:16.946018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:16.947087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:16.947087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:16.947087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:16.947087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 276/530] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:16.947087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e353d-97dc-4b37-80ed-c459c468f6b5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:17.012230 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb64db-d6c5-45d7-af40-e0f8f8ba38d8 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:17.013872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:17.051943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e353d-97dc-4b37-80ed-c459c468f6b5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:17.052703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d922437-a7f6-4c98-ad2a-9d2a7406df00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:17.053599 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d922437-a7f6-4c98-ad2a-9d2a7406df00 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:17.054050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d922437-a7f6-4c98-ad2a-9d2a7406df00 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:17.054513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d922437-a7f6-4c98-ad2a-9d2a7406df00 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:17.055361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d922437-a7f6-4c98-ad2a-9d2a7406df00 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:17.055840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d922437-a7f6-4c98-ad2a-9d2a7406df00 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:17.057486 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d922437-a7f6-4c98-ad2a-9d2a7406df00 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:17.070562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 255/531] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:17.071563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d922437-a7f6-4c98-ad2a-9d2a7406df00 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:17.092639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea26895-4021-4c77-9721-4050c5924d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:17.093271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea26895-4021-4c77-9721-4050c5924d00 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:17.093546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea26895-4021-4c77-9721-4050c5924d00 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:17.093826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea26895-4021-4c77-9721-4050c5924d00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:17.104603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea26895-4021-4c77-9721-4050c5924d00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:17.106135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea26895-4021-4c77-9721-4050c5924d00 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-6ea26895-4021-4c77-9721-4050c5924d00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:17.120685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:17.121869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:17.122872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:17.122872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:17.122872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:17.122872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 277/532] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:17.122872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea26895-4021-4c77-9721-4050c5924d00 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:17.142746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:17.191781 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d922437-a7f6-4c98-ad2a-9d2a7406df00 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:17.228698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664e9a9-d8ce-4a3f-9e8d-47311bb20fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:17.229403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea26895-4021-4c77-9721-4050c5924d00 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:17.230061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664e9a9-d8ce-4a3f-9e8d-47311bb20fc6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:17.230487 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664e9a9-d8ce-4a3f-9e8d-47311bb20fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:17.231019 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664e9a9-d8ce-4a3f-9e8d-47311bb20fc6 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:17.232414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664e9a9-d8ce-4a3f-9e8d-47311bb20fc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:17.232904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664e9a9-d8ce-4a3f-9e8d-47311bb20fc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:17.234521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f664e9a9-d8ce-4a3f-9e8d-47311bb20fc6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:17.246744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:17.249226 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 256/533] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:17.250528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664e9a9-d8ce-4a3f-9e8d-47311bb20fc6 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:17.265388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd84ae7-95cf-4112-abe2-67267005ecc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:17.265951 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd84ae7-95cf-4112-abe2-67267005ecc7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:17.266233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd84ae7-95cf-4112-abe2-67267005ecc7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:17.266546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd84ae7-95cf-4112-abe2-67267005ecc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:17.267302 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd84ae7-95cf-4112-abe2-67267005ecc7 None admin] RBAC: Adding query filter params (role.id=928c1b8875334543ab431e4f7e2acee5, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=4e021a00524341359bc4b5a800a9bbf6) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:17.267603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd84ae7-95cf-4112-abe2-67267005ecc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:17.269094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd84ae7-95cf-4112-abe2-67267005ecc7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:17.279484 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 278/534] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:17 2022] GET /identity/v3/role_assignments?user.id=4e021a00524341359bc4b5a800a9bbf6&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=928c1b8875334543ab431e4f7e2acee5 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:17.280415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd84ae7-95cf-4112-abe2-67267005ecc7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:17.364530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664e9a9-d8ce-4a3f-9e8d-47311bb20fc6 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:17.384954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:17.404130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2915bf5-e9d7-4b40-a28e-11c2ec4c3cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:17.405128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2915bf5-e9d7-4b40-a28e-11c2ec4c3cd7 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:17.405564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2915bf5-e9d7-4b40-a28e-11c2ec4c3cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:17.406083 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2915bf5-e9d7-4b40-a28e-11c2ec4c3cd7 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:17.407020 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2915bf5-e9d7-4b40-a28e-11c2ec4c3cd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:17.407469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2915bf5-e9d7-4b40-a28e-11c2ec4c3cd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:17.409310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2915bf5-e9d7-4b40-a28e-11c2ec4c3cd7 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:17.423755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 257/535] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:17.429403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2915bf5-e9d7-4b40-a28e-11c2ec4c3cd7 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:17.454283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd84ae7-95cf-4112-abe2-67267005ecc7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:17.512856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158ede5-28e6-4f63-85d2-f7624a452cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:17.513503 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158ede5-28e6-4f63-85d2-f7624a452cc6 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:17.513780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158ede5-28e6-4f63-85d2-f7624a452cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:17.514068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158ede5-28e6-4f63-85d2-f7624a452cc6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:17.536543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158ede5-28e6-4f63-85d2-f7624a452cc6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:17.536543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158ede5-28e6-4f63-85d2-f7624a452cc6 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-c158ede5-28e6-4f63-85d2-f7624a452cc6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:17.555939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:17.557026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:17.558069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:17.558069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:17.558069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:17.558069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:17.558069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:17.558069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:17.558069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:17.558069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:17.558573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 279/536] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:17.564780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158ede5-28e6-4f63-85d2-f7624a452cc6 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:17.616835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2915bf5-e9d7-4b40-a28e-11c2ec4c3cd7 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:17.636144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:17.664884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bec34-fc5c-494f-8c4a-0e2c2705e14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:17.665786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bec34-fc5c-494f-8c4a-0e2c2705e14f None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:17.666470 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bec34-fc5c-494f-8c4a-0e2c2705e14f None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:17.666913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4bec34-fc5c-494f-8c4a-0e2c2705e14f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:17.676088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bec34-fc5c-494f-8c4a-0e2c2705e14f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:17.678364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4bec34-fc5c-494f-8c4a-0e2c2705e14f None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-5f4bec34-fc5c-494f-8c4a-0e2c2705e14f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:17.692518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:17.693419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:17.694275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:17.694275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:17.694275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:17.694275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:17.694275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:17.694275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:17.694275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:17.694275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:17.701820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 258/537] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:17.701820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bec34-fc5c-494f-8c4a-0e2c2705e14f None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:17.722945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158ede5-28e6-4f63-85d2-f7624a452cc6 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:17.744598 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:17.811908 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4bec34-fc5c-494f-8c4a-0e2c2705e14f None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:17.827056 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2761b6-ce84-43d5-a2a5-558992cc1a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:17.828002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2761b6-ce84-43d5-a2a5-558992cc1a2b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:17.828528 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2761b6-ce84-43d5-a2a5-558992cc1a2b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:17.828995 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2761b6-ce84-43d5-a2a5-558992cc1a2b None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:17.829896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2761b6-ce84-43d5-a2a5-558992cc1a2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:17.830399 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2761b6-ce84-43d5-a2a5-558992cc1a2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:17.832015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2761b6-ce84-43d5-a2a5-558992cc1a2b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:17.847170 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccdb10-e639-4af5-b580-884ad9318af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:17.847785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccdb10-e639-4af5-b580-884ad9318af5 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:17.848066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccdb10-e639-4af5-b580-884ad9318af5 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:17.848343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccdb10-e639-4af5-b580-884ad9318af5 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:17.849030 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccdb10-e639-4af5-b580-884ad9318af5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:17.849332 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccdb10-e639-4af5-b580-884ad9318af5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:17.850795 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccdb10-e639-4af5-b580-884ad9318af5 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:17.864457 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 259/538] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:17.872954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccdb10-e639-4af5-b580-884ad9318af5 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:17.884122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 280/539] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:17.894809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2761b6-ce84-43d5-a2a5-558992cc1a2b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:17.986968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccdb10-e639-4af5-b580-884ad9318af5 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:17.988816 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:18.047464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7ef2c-e3c4-45a0-a5b9-5d80b924cc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:18.048089 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7ef2c-e3c4-45a0-a5b9-5d80b924cc42 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:18.048369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7ef2c-e3c4-45a0-a5b9-5d80b924cc42 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:18.048643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7ef2c-e3c4-45a0-a5b9-5d80b924cc42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:18.049369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7ef2c-e3c4-45a0-a5b9-5d80b924cc42 None admin] RBAC: Adding query filter params (role.id=928c1b8875334543ab431e4f7e2acee5, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=7103ff9544524393a8c7175998dcae3b) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:18.049677 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7ef2c-e3c4-45a0-a5b9-5d80b924cc42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:18.051151 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7ef2c-e3c4-45a0-a5b9-5d80b924cc42 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:18.057271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 260/540] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:17 2022] GET /identity/v3/role_assignments?user.id=7103ff9544524393a8c7175998dcae3b&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=928c1b8875334543ab431e4f7e2acee5 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:18.089977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2761b6-ce84-43d5-a2a5-558992cc1a2b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:18.091854 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:18.134003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b69e9c-5438-46c6-a52e-c3415fa7787d None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:18.134333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b69e9c-5438-46c6-a52e-c3415fa7787d None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:18.134558 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b69e9c-5438-46c6-a52e-c3415fa7787d None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:18.136235 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 261/541] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:18.142430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c46402-d47b-43cb-bf5e-9b6e26195431 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:18.142714 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c46402-d47b-43cb-bf5e-9b6e26195431 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:18.142997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c46402-d47b-43cb-bf5e-9b6e26195431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:18.153513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f7087a-2541-4889-a035-5eace424ee15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:18.154167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7087a-2541-4889-a035-5eace424ee15 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:18.154446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7087a-2541-4889-a035-5eace424ee15 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:18.154716 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f7087a-2541-4889-a035-5eace424ee15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:18.155433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7087a-2541-4889-a035-5eace424ee15 None admin] RBAC: Adding query filter params (role.id=928c1b8875334543ab431e4f7e2acee5, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=8a2c9eec08b142138569043e3b34c6a8) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:18.155737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7087a-2541-4889-a035-5eace424ee15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:18.157188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f7087a-2541-4889-a035-5eace424ee15 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:18.162095 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-d4c46402-d47b-43cb-bf5e-9b6e26195431 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:18.163015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 281/542] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:17 2022] GET /identity/v3/role_assignments?user.id=8a2c9eec08b142138569043e3b34c6a8&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=928c1b8875334543ab431e4f7e2acee5 => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:18.163610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d4c46402-d47b-43cb-bf5e-9b6e26195431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:18.273122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 262/543] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:34:18.522951 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95555a-641c-4954-b55f-fd1c07ce6855 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:18.523458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95555a-641c-4954-b55f-fd1c07ce6855 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:18.523458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95555a-641c-4954-b55f-fd1c07ce6855 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:18.525048 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 282/544] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:18.531208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e1b3-244b-4006-8a51-c38b43b1f0b1 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:18.531492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e1b3-244b-4006-8a51-c38b43b1f0b1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:18.531765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821e1b3-244b-4006-8a51-c38b43b1f0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:18.550948 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-2821e1b3-244b-4006-8a51-c38b43b1f0b1 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:18.552125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2821e1b3-244b-4006-8a51-c38b43b1f0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:18.567745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95555a-641c-4954-b55f-fd1c07ce6855 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:18.657246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 263/545] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:18.723113 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95555a-641c-4954-b55f-fd1c07ce6855 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:18.767127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519a8a7-eb48-47b8-85f9-e6f19d01210e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:18.767757 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519a8a7-eb48-47b8-85f9-e6f19d01210e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:18.768029 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519a8a7-eb48-47b8-85f9-e6f19d01210e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:18.768291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519a8a7-eb48-47b8-85f9-e6f19d01210e None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:18.783527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519a8a7-eb48-47b8-85f9-e6f19d01210e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-e519a8a7-eb48-47b8-85f9-e6f19d01210e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:18.788533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:18.789385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:18.790190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:18.790190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:18.790190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:18.790190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:18.790190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:18.790190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:18.790190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:18.790190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:18.790190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:18.790190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:18.790190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:18.790190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:18.790190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 283/546] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:18.796065 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821e1b3-244b-4006-8a51-c38b43b1f0b1 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:18.829300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:18.920457 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821e1b3-244b-4006-8a51-c38b43b1f0b1 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:18.964562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3b904-8b49-4ff1-bbc4-10334aafd503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:18.965165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3b904-8b49-4ff1-bbc4-10334aafd503 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:18.965438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3b904-8b49-4ff1-bbc4-10334aafd503 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:18.966222 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3b904-8b49-4ff1-bbc4-10334aafd503 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:18.966897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3b904-8b49-4ff1-bbc4-10334aafd503 None admin] RBAC: Adding query filter params (name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:18.967194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3b904-8b49-4ff1-bbc4-10334aafd503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:18.968564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3b904-8b49-4ff1-bbc4-10334aafd503 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:18.977398 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 264/547] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:18.983235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519a8a7-eb48-47b8-85f9-e6f19d01210e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:19.026731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:19.061138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3b904-8b49-4ff1-bbc4-10334aafd503 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:19.122613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519a8a7-eb48-47b8-85f9-e6f19d01210e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:19.172063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a6106-9936-492e-8fed-1eb19d2aecbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:19.172704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a6106-9936-492e-8fed-1eb19d2aecbb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:19.172978 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a6106-9936-492e-8fed-1eb19d2aecbb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:19.173245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a6106-9936-492e-8fed-1eb19d2aecbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:19.194750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a6106-9936-492e-8fed-1eb19d2aecbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:19.196204 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285a6106-9936-492e-8fed-1eb19d2aecbb None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:19.223508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-285a6106-9936-492e-8fed-1eb19d2aecbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:19.231639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:19.232547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:19.238379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:19.238379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:19.238379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:19.238379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 284/548] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:19.241514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a6106-9936-492e-8fed-1eb19d2aecbb None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:19.259774 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3b904-8b49-4ff1-bbc4-10334aafd503 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:19.323671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8b0b4-e5df-4419-bdea-445c3500d7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:19.324293 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8b0b4-e5df-4419-bdea-445c3500d7ab None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:19.324507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8b0b4-e5df-4419-bdea-445c3500d7ab None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:19.324784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8b0b4-e5df-4419-bdea-445c3500d7ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:19.350037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8b0b4-e5df-4419-bdea-445c3500d7ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-1fa8b0b4-e5df-4419-bdea-445c3500d7ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:19.355590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:19.357033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:19.358003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:19.358003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:19.358003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:19.358003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:19.358003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:19.358003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:19.358003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:19.358003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:19.358003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:19.358003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:19.358003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:19.358003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:19.359363 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 265/549] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:19.366431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8b0b4-e5df-4419-bdea-445c3500d7ab None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:19.394153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:19.427880 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a6106-9936-492e-8fed-1eb19d2aecbb None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:19.492155 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe45de-180b-468a-b420-1b0a9c09f377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:19.492771 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe45de-180b-468a-b420-1b0a9c09f377 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:19.493046 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe45de-180b-468a-b420-1b0a9c09f377 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:19.493314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe45de-180b-468a-b420-1b0a9c09f377 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:19.496889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe45de-180b-468a-b420-1b0a9c09f377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:19.496889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe45de-180b-468a-b420-1b0a9c09f377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:19.496889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe45de-180b-468a-b420-1b0a9c09f377 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:19.520632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 285/550] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:19.529953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe45de-180b-468a-b420-1b0a9c09f377 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:19.556014 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8b0b4-e5df-4419-bdea-445c3500d7ab None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:19.622676 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd5742-74ff-49ce-9fdb-2c1d86f743c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:19.623570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd5742-74ff-49ce-9fdb-2c1d86f743c1 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:19.624022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd5742-74ff-49ce-9fdb-2c1d86f743c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:19.624460 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd5742-74ff-49ce-9fdb-2c1d86f743c1 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:19.625288 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd5742-74ff-49ce-9fdb-2c1d86f743c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:19.625779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd5742-74ff-49ce-9fdb-2c1d86f743c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:19.627349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd5742-74ff-49ce-9fdb-2c1d86f743c1 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:19.645273 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 266/551] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:19.650309 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd5742-74ff-49ce-9fdb-2c1d86f743c1 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:19.659544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:19.676718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe45de-180b-468a-b420-1b0a9c09f377 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:19.715120 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a718622-2693-4026-bf92-36de0c3d4a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:19.716667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a718622-2693-4026-bf92-36de0c3d4a66 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:19.717201 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a718622-2693-4026-bf92-36de0c3d4a66 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:19.717671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a718622-2693-4026-bf92-36de0c3d4a66 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:19.726381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a718622-2693-4026-bf92-36de0c3d4a66 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:19.728564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a718622-2693-4026-bf92-36de0c3d4a66 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-9a718622-2693-4026-bf92-36de0c3d4a66 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:19.741513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:19.742516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:19.743430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:19.743430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:19.743430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:19.743430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:19.743430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:19.743430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:19.743430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:19.743430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:19.743430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 19:34:19.743430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:19.746551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 286/552] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:34:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:19.750011 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:19.753000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a718622-2693-4026-bf92-36de0c3d4a66 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:19.889690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd5742-74ff-49ce-9fdb-2c1d86f743c1 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:19.918789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a718622-2693-4026-bf92-36de0c3d4a66 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:19.959211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987bc229-d093-4afc-a86a-354c2b9ae758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:19.960202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bc229-d093-4afc-a86a-354c2b9ae758 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:19.960628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bc229-d093-4afc-a86a-354c2b9ae758 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:19.960972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987bc229-d093-4afc-a86a-354c2b9ae758 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:19.961941 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987bc229-d093-4afc-a86a-354c2b9ae758 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:19.962297 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987bc229-d093-4afc-a86a-354c2b9ae758 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:19.963944 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987bc229-d093-4afc-a86a-354c2b9ae758 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:19.977423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a341b-8758-4771-a6ca-d6d7ade2a386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:19.978103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a341b-8758-4771-a6ca-d6d7ade2a386 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:19.978385 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a341b-8758-4771-a6ca-d6d7ade2a386 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:19.978655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a341b-8758-4771-a6ca-d6d7ade2a386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:19.983789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 287/553] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Sep 14 19:34:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:19.989311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987bc229-d093-4afc-a86a-354c2b9ae758 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:19.995990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a341b-8758-4771-a6ca-d6d7ade2a386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:20.000407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a341b-8758-4771-a6ca-d6d7ade2a386 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-5d7a341b-8758-4771-a6ca-d6d7ade2a386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:20.023857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:20.024902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:20.026343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:20.026343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:20.026343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:20.026343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 267/554] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:20.026580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc27a84-2d29-4a9e-ab6d-1e6429c0fb83 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:20.026768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc27a84-2d29-4a9e-ab6d-1e6429c0fb83 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:20.026991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc27a84-2d29-4a9e-ab6d-1e6429c0fb83 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:20.029839 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 268/555] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:20.033875 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc27a84-2d29-4a9e-ab6d-1e6429c0fb83 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:20.082597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:20.115177 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:20.129045 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987bc229-d093-4afc-a86a-354c2b9ae758 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:20.167951 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd0425-50ed-41da-bb6f-12609317f9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:20.168661 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd0425-50ed-41da-bb6f-12609317f9e9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:20.169468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd0425-50ed-41da-bb6f-12609317f9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:20.169742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd0425-50ed-41da-bb6f-12609317f9e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:20.170847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc27a84-2d29-4a9e-ab6d-1e6429c0fb83 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:20.217597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0865aa-d8c9-4faf-a638-b16cf044fdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:20.218254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0865aa-d8c9-4faf-a638-b16cf044fdf5 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:20.218536 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0865aa-d8c9-4faf-a638-b16cf044fdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:20.218866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0865aa-d8c9-4faf-a638-b16cf044fdf5 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:20.219570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0865aa-d8c9-4faf-a638-b16cf044fdf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:20.219876 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0865aa-d8c9-4faf-a638-b16cf044fdf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:20.221770 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0865aa-d8c9-4faf-a638-b16cf044fdf5 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:20.244008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 269/556] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:20.244008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b182c5-345d-4dd3-ab81-ca960e32df76 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:20.244008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b182c5-345d-4dd3-ab81-ca960e32df76 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:20.244008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b182c5-345d-4dd3-ab81-ca960e32df76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:20.255262 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd0425-50ed-41da-bb6f-12609317f9e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:20.257000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd0425-50ed-41da-bb6f-12609317f9e9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:20.264764 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-39b182c5-345d-4dd3-ab81-ca960e32df76 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:20.267327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-39b182c5-345d-4dd3-ab81-ca960e32df76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:20.320561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:20.339589 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 270/557] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:20.340560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b182c5-345d-4dd3-ab81-ca960e32df76 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:20.369145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 288/558] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:34:20.383234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd0425-50ed-41da-bb6f-12609317f9e9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:20.398595 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:20.455974 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b182c5-345d-4dd3-ab81-ca960e32df76 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:20.495870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b8261-2fc0-4110-b392-08fbcbd483ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:20.499977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b8261-2fc0-4110-b392-08fbcbd483ce None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:20.499977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b8261-2fc0-4110-b392-08fbcbd483ce None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:20.499977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1b8261-2fc0-4110-b392-08fbcbd483ce None admin] PATH_INFO: `/v3/users/nova` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:20.500241 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd0425-50ed-41da-bb6f-12609317f9e9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:20.506277 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b8261-2fc0-4110-b392-08fbcbd483ce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:20.508113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1b8261-2fc0-4110-b392-08fbcbd483ce None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-cf1b8261-2fc0-4110-b392-08fbcbd483ce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:20.520962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:20.522197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:20.523155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:20.523155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:20.523155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:20.523155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:20.523155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:20.523155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:20.523155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:20.523155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:20.523155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:34:20.523155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:20.526303 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 271/559] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 19:34:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:20.539235 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b8261-2fc0-4110-b392-08fbcbd483ce None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:20.544568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a430c2-9e10-4dad-9ecc-a08b8025878e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:20.545204 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a430c2-9e10-4dad-9ecc-a08b8025878e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:20.545479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a430c2-9e10-4dad-9ecc-a08b8025878e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:20.545744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a430c2-9e10-4dad-9ecc-a08b8025878e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:20.566501 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a430c2-9e10-4dad-9ecc-a08b8025878e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:20.568219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a430c2-9e10-4dad-9ecc-a08b8025878e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-54a430c2-9e10-4dad-9ecc-a08b8025878e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:20.593557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:20.594594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:20.595494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:20.595494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:20.595494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:20.595494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 289/560] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:20.602750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a430c2-9e10-4dad-9ecc-a08b8025878e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:20.622705 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:20.690556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1b8261-2fc0-4110-b392-08fbcbd483ce None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:20.701389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:20.731454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0beab6-e001-418d-9796-69ab3583c700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:20.732060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0beab6-e001-418d-9796-69ab3583c700 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:20.732333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0beab6-e001-418d-9796-69ab3583c700 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:20.732598 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0beab6-e001-418d-9796-69ab3583c700 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:20.733479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0beab6-e001-418d-9796-69ab3583c700 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:20.734268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0beab6-e001-418d-9796-69ab3583c700 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:20.736147 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0beab6-e001-418d-9796-69ab3583c700 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:20.758090 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 272/561] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 19:34:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:20.759117 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0beab6-e001-418d-9796-69ab3583c700 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:20.772123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a430c2-9e10-4dad-9ecc-a08b8025878e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:20.811899 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbba64-60c8-4418-88a4-228dcb1977c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:20.812504 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbba64-60c8-4418-88a4-228dcb1977c8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:20.812786 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbba64-60c8-4418-88a4-228dcb1977c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:20.813059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dbba64-60c8-4418-88a4-228dcb1977c8 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:20.814132 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dbba64-60c8-4418-88a4-228dcb1977c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:20.814448 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dbba64-60c8-4418-88a4-228dcb1977c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:20.815811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dbba64-60c8-4418-88a4-228dcb1977c8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:20.828187 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 290/562] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:20.828973 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbba64-60c8-4418-88a4-228dcb1977c8 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:20.861064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0beab6-e001-418d-9796-69ab3583c700 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:20.894269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:20.897441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c84c9-0994-4daa-ad3f-3f71f779f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:20.898087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c84c9-0994-4daa-ad3f-3f71f779f8f0 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:20.898362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c84c9-0994-4daa-ad3f-3f71f779f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:20.898626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c84c9-0994-4daa-ad3f-3f71f779f8f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:20.908627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c84c9-0994-4daa-ad3f-3f71f779f8f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-309c84c9-0994-4daa-ad3f-3f71f779f8f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:20.913414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:20.914580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:20.915504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:20.915504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:20.915504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:20.915504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:20.915504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:20.915504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:20.915504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:20.915504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:20.915504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:20.915504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:20.915504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:20.915504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:20.915504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 273/563] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:20.916247 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554b386-90cc-4cc4-97ae-3a9f8f04488f None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:20.916471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554b386-90cc-4cc4-97ae-3a9f8f04488f None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:20.916680 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554b386-90cc-4cc4-97ae-3a9f8f04488f None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:20.917914 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 274/564] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:20.920794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f896a-4bc3-46fd-a853-524b0167cb83 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:20.921072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f896a-4bc3-46fd-a853-524b0167cb83 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:20.921403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f896a-4bc3-46fd-a853-524b0167cb83 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:20.923401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 275/565] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:20.924107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f896a-4bc3-46fd-a853-524b0167cb83 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:20.932318 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dbba64-60c8-4418-88a4-228dcb1977c8 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:20.951870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:20.965087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a10c8a-b560-437b-9d4f-7d3f065e02aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:20.965767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a10c8a-b560-437b-9d4f-7d3f065e02aa None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:20.966070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a10c8a-b560-437b-9d4f-7d3f065e02aa None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:20.966341 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a10c8a-b560-437b-9d4f-7d3f065e02aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:20.976267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a10c8a-b560-437b-9d4f-7d3f065e02aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:20.977773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a10c8a-b560-437b-9d4f-7d3f065e02aa None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-c4a10c8a-b560-437b-9d4f-7d3f065e02aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:20.992351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:20.993280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:20.994191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:20.994191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:20.994191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:20.994191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 291/566] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:20.994191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a10c8a-b560-437b-9d4f-7d3f065e02aa None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:21.029946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f896a-4bc3-46fd-a853-524b0167cb83 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:21.062885 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bb0f5-ec27-4c9e-b1a1-2fb9b9b09fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:21.063499 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bb0f5-ec27-4c9e-b1a1-2fb9b9b09fa7 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:21.063769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bb0f5-ec27-4c9e-b1a1-2fb9b9b09fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:21.064034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bb0f5-ec27-4c9e-b1a1-2fb9b9b09fa7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:21.070744 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bb0f5-ec27-4c9e-b1a1-2fb9b9b09fa7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:21.072173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bb0f5-ec27-4c9e-b1a1-2fb9b9b09fa7 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-8b1bb0f5-ec27-4c9e-b1a1-2fb9b9b09fa7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:21.083567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:21.084522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:21.090878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:21.090878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:21.090878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:21.090878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:21.090878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:21.090878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:21.090878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:21.090878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:21.090878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 276/567] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:21.090878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d3833-c110-4e89-a952-045947e597b9 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:21.090878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d3833-c110-4e89-a952-045947e597b9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:21.091516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d3833-c110-4e89-a952-045947e597b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:21.091585 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a10c8a-b560-437b-9d4f-7d3f065e02aa None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:21.103958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-5e1d3833-c110-4e89-a952-045947e597b9 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:21.105128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-5e1d3833-c110-4e89-a952-045947e597b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:21.110533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:21.123947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26107e-f8ca-443f-a1a9-a8c2c0eee922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:21.132002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26107e-f8ca-443f-a1a9-a8c2c0eee922 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:21.132002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26107e-f8ca-443f-a1a9-a8c2c0eee922 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:21.132002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26107e-f8ca-443f-a1a9-a8c2c0eee922 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:21.132002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26107e-f8ca-443f-a1a9-a8c2c0eee922 None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:21.132002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26107e-f8ca-443f-a1a9-a8c2c0eee922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:21.132002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26107e-f8ca-443f-a1a9-a8c2c0eee922 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:21.135992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 292/568] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:21.137190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302207fb-b4de-4e5c-871e-bc14ee34734a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:21.138032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302207fb-b4de-4e5c-871e-bc14ee34734a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:21.138322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302207fb-b4de-4e5c-871e-bc14ee34734a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:21.156970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-302207fb-b4de-4e5c-871e-bc14ee34734a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:21.158198 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-302207fb-b4de-4e5c-871e-bc14ee34734a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:21.173068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 277/569] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:21.173930 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d3833-c110-4e89-a952-045947e597b9 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:21.194322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:21.229309 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 293/570] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:21.230101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302207fb-b4de-4e5c-871e-bc14ee34734a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:21.282164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d3833-c110-4e89-a952-045947e597b9 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:21.329236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5e698-dd89-4420-a8f4-7d5d607d1dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:21.329874 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5e698-dd89-4420-a8f4-7d5d607d1dfe None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:21.330191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5e698-dd89-4420-a8f4-7d5d607d1dfe None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:21.330465 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5e698-dd89-4420-a8f4-7d5d607d1dfe None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:21.331485 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5e698-dd89-4420-a8f4-7d5d607d1dfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:21.331789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5e698-dd89-4420-a8f4-7d5d607d1dfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:21.333149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5e698-dd89-4420-a8f4-7d5d607d1dfe None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:21.347076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302207fb-b4de-4e5c-871e-bc14ee34734a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:21.349941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 278/571] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:21.354726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5e698-dd89-4420-a8f4-7d5d607d1dfe None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:21.361446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:21.383005 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fe242-ddf1-4340-b707-3fbb1e89ba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:21.383908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fe242-ddf1-4340-b707-3fbb1e89ba6e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:21.384418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fe242-ddf1-4340-b707-3fbb1e89ba6e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:21.384873 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2fe242-ddf1-4340-b707-3fbb1e89ba6e None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:21.385771 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fe242-ddf1-4340-b707-3fbb1e89ba6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:21.386269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fe242-ddf1-4340-b707-3fbb1e89ba6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:21.387935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2fe242-ddf1-4340-b707-3fbb1e89ba6e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:21.399702 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 294/572] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:21.400731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fe242-ddf1-4340-b707-3fbb1e89ba6e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:21.452359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:21.474117 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5e698-dd89-4420-a8f4-7d5d607d1dfe None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:21.510594 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f885e-68b5-4425-bc70-36ba61841129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:21.511555 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f885e-68b5-4425-bc70-36ba61841129 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:21.512068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f885e-68b5-4425-bc70-36ba61841129 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:21.512701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f885e-68b5-4425-bc70-36ba61841129 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:21.517696 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2fe242-ddf1-4340-b707-3fbb1e89ba6e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:21.523941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f885e-68b5-4425-bc70-36ba61841129 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:21.525383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f885e-68b5-4425-bc70-36ba61841129 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-019f885e-68b5-4425-bc70-36ba61841129 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:21.540254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:21.541408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:21.542481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:21.542481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:21.542481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:21.542481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 279/573] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:21.542712 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f885e-68b5-4425-bc70-36ba61841129 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:21.556877 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ebb90-bfd1-4003-bb72-986281f7688f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:21.557512 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ebb90-bfd1-4003-bb72-986281f7688f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:21.557790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ebb90-bfd1-4003-bb72-986281f7688f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:21.558104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ebb90-bfd1-4003-bb72-986281f7688f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:21.564665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ebb90-bfd1-4003-bb72-986281f7688f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:21.566123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ebb90-bfd1-4003-bb72-986281f7688f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-0c4ebb90-bfd1-4003-bb72-986281f7688f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:21.578067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:21.579101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:21.580003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:21.580003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:21.580003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:21.580003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:21.580003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:21.580003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:21.580003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:21.580003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:21.580003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 295/574] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:21.580003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ebb90-bfd1-4003-bb72-986281f7688f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:21.611932 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:21.644139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f885e-68b5-4425-bc70-36ba61841129 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:21.669365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:21.677025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ebb90-bfd1-4003-bb72-986281f7688f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:21.683470 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14102f-db6f-42bd-a436-7efd4d546421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:21.683470 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14102f-db6f-42bd-a436-7efd4d546421 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:21.683470 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14102f-db6f-42bd-a436-7efd4d546421 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:21.683470 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14102f-db6f-42bd-a436-7efd4d546421 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:21.683816 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14102f-db6f-42bd-a436-7efd4d546421 None admin] RBAC: Adding query filter params (role.id=928c1b8875334543ab431e4f7e2acee5, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=e3d2f1ab1bfd435183e33168400c0b17) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:21.683816 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14102f-db6f-42bd-a436-7efd4d546421 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:21.683816 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14102f-db6f-42bd-a436-7efd4d546421 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:21.685819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 280/575] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:21 2022] GET /identity/v3/role_assignments?user.id=e3d2f1ab1bfd435183e33168400c0b17&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=928c1b8875334543ab431e4f7e2acee5 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:21.686597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14102f-db6f-42bd-a436-7efd4d546421 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:21.714165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa5c16-f4d3-44b7-ab10-c9dc079714ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:21.715149 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa5c16-f4d3-44b7-ab10-c9dc079714ae None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:21.715676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa5c16-f4d3-44b7-ab10-c9dc079714ae None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:21.716169 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faa5c16-f4d3-44b7-ab10-c9dc079714ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:21.728282 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faa5c16-f4d3-44b7-ab10-c9dc079714ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-8faa5c16-f4d3-44b7-ab10-c9dc079714ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:21.735355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:21.736368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:21.737211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:21.737211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:21.737211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:21.737211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:21.737211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:21.737211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:21.737211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:21.737211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:21.737211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:21.737211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:21.737211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:21.737211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:21.737211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 296/576] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:21.737211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa5c16-f4d3-44b7-ab10-c9dc079714ae None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:21.811959 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14102f-db6f-42bd-a436-7efd4d546421 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:21.842639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:21.850802 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf75260-147b-4d73-9008-cee6806eeee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:21.855456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf75260-147b-4d73-9008-cee6806eeee6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:21.855577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faa5c16-f4d3-44b7-ab10-c9dc079714ae None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:21.855696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf75260-147b-4d73-9008-cee6806eeee6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:21.855696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf75260-147b-4d73-9008-cee6806eeee6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:21.865669 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf75260-147b-4d73-9008-cee6806eeee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-9cf75260-147b-4d73-9008-cee6806eeee6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:21.871729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:21.872891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:21.878641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:21.878641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:21.878641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:21.878641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:21.878641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:21.878641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:21.878641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:21.878641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:21.878641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:21.878641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:21.878641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:21.878641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:21.878641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 281/577] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:21.878641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf75260-147b-4d73-9008-cee6806eeee6 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:21.899300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79590361-ccd6-4b43-ae2a-57ed4c1ebd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:21.899910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79590361-ccd6-4b43-ae2a-57ed4c1ebd27 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:21.900186 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79590361-ccd6-4b43-ae2a-57ed4c1ebd27 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:21.900457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79590361-ccd6-4b43-ae2a-57ed4c1ebd27 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:21.901090 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79590361-ccd6-4b43-ae2a-57ed4c1ebd27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:21.901392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79590361-ccd6-4b43-ae2a-57ed4c1ebd27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:21.902802 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79590361-ccd6-4b43-ae2a-57ed4c1ebd27 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:21.915099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 297/578] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:21.916145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79590361-ccd6-4b43-ae2a-57ed4c1ebd27 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:21.935062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:21.982938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf75260-147b-4d73-9008-cee6806eeee6 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:22.016860 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902da7d-23b6-4c5e-bd6a-2833ca2d1af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:22.017466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902da7d-23b6-4c5e-bd6a-2833ca2d1af7 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:22.017738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902da7d-23b6-4c5e-bd6a-2833ca2d1af7 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:22.018033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902da7d-23b6-4c5e-bd6a-2833ca2d1af7 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:22.018706 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a902da7d-23b6-4c5e-bd6a-2833ca2d1af7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:22.019004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a902da7d-23b6-4c5e-bd6a-2833ca2d1af7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:22.020438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a902da7d-23b6-4c5e-bd6a-2833ca2d1af7 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:22.023824 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79590361-ccd6-4b43-ae2a-57ed4c1ebd27 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:22.033003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 282/579] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:22.033618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902da7d-23b6-4c5e-bd6a-2833ca2d1af7 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:22.066598 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594759f-5393-4c2d-b8c8-d33f8307f15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:22.067131 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594759f-5393-4c2d-b8c8-d33f8307f15d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:22.067668 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594759f-5393-4c2d-b8c8-d33f8307f15d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:22.067942 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594759f-5393-4c2d-b8c8-d33f8307f15d None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:22.068760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594759f-5393-4c2d-b8c8-d33f8307f15d None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:22.069871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594759f-5393-4c2d-b8c8-d33f8307f15d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:22.071232 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594759f-5393-4c2d-b8c8-d33f8307f15d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:22.080416 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 298/580] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:22.081222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594759f-5393-4c2d-b8c8-d33f8307f15d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:22.098506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:22.139537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902da7d-23b6-4c5e-bd6a-2833ca2d1af7 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:22.153643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:22.174865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a8671-c646-4628-b013-330a7eebf1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:22.175529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a8671-c646-4628-b013-330a7eebf1b6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:22.175808 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a8671-c646-4628-b013-330a7eebf1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:22.176078 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a8671-c646-4628-b013-330a7eebf1b6 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:22.176757 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a8671-c646-4628-b013-330a7eebf1b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:22.177058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a8671-c646-4628-b013-330a7eebf1b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:22.178469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a8671-c646-4628-b013-330a7eebf1b6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:22.190655 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594759f-5393-4c2d-b8c8-d33f8307f15d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:22.191304 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 283/581] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:22.192148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a8671-c646-4628-b013-330a7eebf1b6 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:22.225365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5218f-9ab0-4386-8eb9-eb584ea7cb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:22.226017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5218f-9ab0-4386-8eb9-eb584ea7cb73 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:22.226291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5218f-9ab0-4386-8eb9-eb584ea7cb73 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:22.226558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5218f-9ab0-4386-8eb9-eb584ea7cb73 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:22.234549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5218f-9ab0-4386-8eb9-eb584ea7cb73 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:22.235970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5218f-9ab0-4386-8eb9-eb584ea7cb73 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-69b5218f-9ab0-4386-8eb9-eb584ea7cb73 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:22.247988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:22.249220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:22.250603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:22.250603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:22.250603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:22.250603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:22.250603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:22.250603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:22.250603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:22.250603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:22.250603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 19:34:22.250603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:22.250603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 299/582] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:22.251441 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5218f-9ab0-4386-8eb9-eb584ea7cb73 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:22.300521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a8671-c646-4628-b013-330a7eebf1b6 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:22.329599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:22.333921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090168a1-2e93-4c49-981e-adc489717873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:22.334703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090168a1-2e93-4c49-981e-adc489717873 None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:22.335100 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090168a1-2e93-4c49-981e-adc489717873 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:22.335435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090168a1-2e93-4c49-981e-adc489717873 None admin] PATH_INFO: `/v3/projects/6bfc6c1c443849038af35577daf34088/users/2a4284b59f804dc59674d621d87634bc/roles/928c1b8875334543ab431e4f7e2acee5` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:22.350126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090168a1-2e93-4c49-981e-adc489717873 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bfc6c1c443849038af35577daf34088, user_id=2a4284b59f804dc59674d621d87634bc, role_id=928c1b8875334543ab431e4f7e2acee5)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:22.351580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090168a1-2e93-4c49-981e-adc489717873 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:22.355568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5218f-9ab0-4386-8eb9-eb584ea7cb73 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:22.389127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4962118-fdf5-4584-afd3-f07898e1099c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:22.389752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4962118-fdf5-4584-afd3-f07898e1099c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:22.390076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4962118-fdf5-4584-afd3-f07898e1099c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:22.390343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4962118-fdf5-4584-afd3-f07898e1099c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:22.400875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4962118-fdf5-4584-afd3-f07898e1099c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:22.409762 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4962118-fdf5-4584-afd3-f07898e1099c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-d4962118-fdf5-4584-afd3-f07898e1099c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:22.416843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:22.418054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:22.419215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:22.419215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:22.419215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:22.421942 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 300/583] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:22.423278 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4962118-fdf5-4584-afd3-f07898e1099c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:22.434676 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 284/584] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:34:22 2022] PUT /identity/v3/projects/6bfc6c1c443849038af35577daf34088/users/2a4284b59f804dc59674d621d87634bc/roles/928c1b8875334543ab431e4f7e2acee5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:34:22.435774 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090168a1-2e93-4c49-981e-adc489717873 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:22.471339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:22.560153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4962118-fdf5-4584-afd3-f07898e1099c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:22.596355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090168a1-2e93-4c49-981e-adc489717873 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:22.597509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:22.614645 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b14d84-97f6-4c35-8532-a340714be254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:22.615175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b14d84-97f6-4c35-8532-a340714be254 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:22.616238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b14d84-97f6-4c35-8532-a340714be254 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:22.617439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b14d84-97f6-4c35-8532-a340714be254 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:22.632027 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b14d84-97f6-4c35-8532-a340714be254 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:22.633496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b14d84-97f6-4c35-8532-a340714be254 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-28b14d84-97f6-4c35-8532-a340714be254 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:22.648458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:22.649592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:22.650789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:22.650789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:22.650789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:22.650994 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20620942-2142-44c4-a66e-ee45b5b803f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:22.650994 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20620942-2142-44c4-a66e-ee45b5b803f4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:22.650994 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20620942-2142-44c4-a66e-ee45b5b803f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:22.651250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 301/585] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:22.651338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20620942-2142-44c4-a66e-ee45b5b803f4 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:22.651427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b14d84-97f6-4c35-8532-a340714be254 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:22.651520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20620942-2142-44c4-a66e-ee45b5b803f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:22.651520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20620942-2142-44c4-a66e-ee45b5b803f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:22.652665 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20620942-2142-44c4-a66e-ee45b5b803f4 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:22.672259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 285/586] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Sep 14 19:34:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:22.672899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20620942-2142-44c4-a66e-ee45b5b803f4 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:22.769587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b14d84-97f6-4c35-8532-a340714be254 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:22.784272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:22.806768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87dd31a-64e3-4b32-836f-0859a695ca4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:22.807687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dd31a-64e3-4b32-836f-0859a695ca4e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:22.807945 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20620942-2142-44c4-a66e-ee45b5b803f4 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:22.808273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dd31a-64e3-4b32-836f-0859a695ca4e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:22.808682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dd31a-64e3-4b32-836f-0859a695ca4e None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:22.810006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87dd31a-64e3-4b32-836f-0859a695ca4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:22.810503 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87dd31a-64e3-4b32-836f-0859a695ca4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:22.812077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87dd31a-64e3-4b32-836f-0859a695ca4e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:22.827156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 302/587] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:22.829747 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87dd31a-64e3-4b32-836f-0859a695ca4e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:22.853570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57956567-555a-415c-9ff3-e5d22c2ca83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:22.856883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57956567-555a-415c-9ff3-e5d22c2ca83c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:22.856883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57956567-555a-415c-9ff3-e5d22c2ca83c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:22.856883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57956567-555a-415c-9ff3-e5d22c2ca83c None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:22.856883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57956567-555a-415c-9ff3-e5d22c2ca83c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:22.856883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57956567-555a-415c-9ff3-e5d22c2ca83c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:22.857330 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57956567-555a-415c-9ff3-e5d22c2ca83c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:22.873505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 286/588] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:22.874482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57956567-555a-415c-9ff3-e5d22c2ca83c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:22.885278 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:22.961452 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87dd31a-64e3-4b32-836f-0859a695ca4e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:23.001026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70593bac-8bcd-42aa-b842-81d21906a6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:23.001778 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70593bac-8bcd-42aa-b842-81d21906a6a0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:23.002084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70593bac-8bcd-42aa-b842-81d21906a6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:23.002595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70593bac-8bcd-42aa-b842-81d21906a6a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:23.005940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57956567-555a-415c-9ff3-e5d22c2ca83c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:23.013368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70593bac-8bcd-42aa-b842-81d21906a6a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:23.015064 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70593bac-8bcd-42aa-b842-81d21906a6a0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-70593bac-8bcd-42aa-b842-81d21906a6a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:23.031749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:23.032893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:23.033999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:23.033999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:23.033999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:23.033999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 303/589] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:23.034298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70593bac-8bcd-42aa-b842-81d21906a6a0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:23.054701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ef286-3b54-491f-830e-53af615f6a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:23.055624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ef286-3b54-491f-830e-53af615f6a79 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:23.056038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ef286-3b54-491f-830e-53af615f6a79 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:23.056520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ef286-3b54-491f-830e-53af615f6a79 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:23.065310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ef286-3b54-491f-830e-53af615f6a79 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:23.067001 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ef286-3b54-491f-830e-53af615f6a79 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-7a3ef286-3b54-491f-830e-53af615f6a79 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:23.079397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:23.080612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:23.081693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:23.081693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:23.081693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:23.081693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:23.081693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:23.081693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:23.081693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:23.081693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:23.081693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 19:34:23.081693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:23.088146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 287/590] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:23.089197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ef286-3b54-491f-830e-53af615f6a79 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:23.092721 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:23.139101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:23.149371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70593bac-8bcd-42aa-b842-81d21906a6a0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:23.187574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372bd66-4a21-4629-874f-cc4fe685a164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:23.188214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372bd66-4a21-4629-874f-cc4fe685a164 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:23.188838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372bd66-4a21-4629-874f-cc4fe685a164 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:23.189322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372bd66-4a21-4629-874f-cc4fe685a164 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:23.195910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ef286-3b54-491f-830e-53af615f6a79 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:23.197645 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372bd66-4a21-4629-874f-cc4fe685a164 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:23.199231 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372bd66-4a21-4629-874f-cc4fe685a164 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-0372bd66-4a21-4629-874f-cc4fe685a164 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:23.211314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:23.212490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:23.213577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:23.213577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:23.213577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:23.213577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:23.213577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:23.213577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:23.213577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:23.213577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:23.213577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 19:34:23.213577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:23.213577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 304/591] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:23.213577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372bd66-4a21-4629-874f-cc4fe685a164 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:23.235385 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1bad0-db8b-4b0d-b137-348726441ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:23.236277 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1bad0-db8b-4b0d-b137-348726441ec3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:23.236726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1bad0-db8b-4b0d-b137-348726441ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:23.237138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1bad0-db8b-4b0d-b137-348726441ec3 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:23.238004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1bad0-db8b-4b0d-b137-348726441ec3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:23.238523 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1bad0-db8b-4b0d-b137-348726441ec3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:23.240133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1bad0-db8b-4b0d-b137-348726441ec3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:23.258051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 288/592] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:23.259149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1bad0-db8b-4b0d-b137-348726441ec3 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:23.321925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:23.325572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372bd66-4a21-4629-874f-cc4fe685a164 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:23.363519 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785257c8-9196-4929-af07-cca82126042e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:23.364305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785257c8-9196-4929-af07-cca82126042e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:23.364517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785257c8-9196-4929-af07-cca82126042e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:23.364795 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785257c8-9196-4929-af07-cca82126042e None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:23.365515 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785257c8-9196-4929-af07-cca82126042e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:23.365842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785257c8-9196-4929-af07-cca82126042e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:23.367587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785257c8-9196-4929-af07-cca82126042e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:23.370918 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1bad0-db8b-4b0d-b137-348726441ec3 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:23.393262 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 305/593] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Sep 14 19:34:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:23.393262 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785257c8-9196-4929-af07-cca82126042e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:23.401993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:23.416080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d499989-2a75-442a-8954-77f998d20d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:23.417032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d499989-2a75-442a-8954-77f998d20d65 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:23.417459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d499989-2a75-442a-8954-77f998d20d65 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:23.417936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d499989-2a75-442a-8954-77f998d20d65 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:23.418826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d499989-2a75-442a-8954-77f998d20d65 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:23.419288 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d499989-2a75-442a-8954-77f998d20d65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:23.421038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d499989-2a75-442a-8954-77f998d20d65 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:23.440316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 289/594] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Sep 14 19:34:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:23.441362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d499989-2a75-442a-8954-77f998d20d65 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:23.509187 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785257c8-9196-4929-af07-cca82126042e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:23.552526 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d499989-2a75-442a-8954-77f998d20d65 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:23.553027 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb13634-4d09-4c0d-a62f-27058b04610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:23.553671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb13634-4d09-4c0d-a62f-27058b04610d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:23.553997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb13634-4d09-4c0d-a62f-27058b04610d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:23.554265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb13634-4d09-4c0d-a62f-27058b04610d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:23.563004 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb13634-4d09-4c0d-a62f-27058b04610d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:23.564953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb13634-4d09-4c0d-a62f-27058b04610d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:23.578010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-4eb13634-4d09-4c0d-a62f-27058b04610d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:23.581313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:23.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:23.583136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:23.583136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:23.583136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:23.583136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:23.583136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:23.583136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:23.583136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:23.583136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:23.584569 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 306/595] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:23.585560 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb13634-4d09-4c0d-a62f-27058b04610d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:23.601045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55cd6c-1509-4172-8d64-955b271a4a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:23.601638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55cd6c-1509-4172-8d64-955b271a4a5d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:23.601955 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55cd6c-1509-4172-8d64-955b271a4a5d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:23.602278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55cd6c-1509-4172-8d64-955b271a4a5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:23.613882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55cd6c-1509-4172-8d64-955b271a4a5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:23.615381 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd55cd6c-1509-4172-8d64-955b271a4a5d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-bd55cd6c-1509-4172-8d64-955b271a4a5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:23.631026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:23.631997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:23.632905 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:23.632905 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:23.632905 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:23.636793 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 290/596] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:23.637706 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55cd6c-1509-4172-8d64-955b271a4a5d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:23.672747 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:23.716199 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb13634-4d09-4c0d-a62f-27058b04610d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:23.744490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55cd6c-1509-4172-8d64-955b271a4a5d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:23.755982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfdfc1-be79-4b68-8ffb-dda32964a9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:23.756919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfdfc1-be79-4b68-8ffb-dda32964a9e0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:23.757418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfdfc1-be79-4b68-8ffb-dda32964a9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:23.758166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bfdfc1-be79-4b68-8ffb-dda32964a9e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:23.774179 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bfdfc1-be79-4b68-8ffb-dda32964a9e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:23.775644 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bfdfc1-be79-4b68-8ffb-dda32964a9e0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:23.789751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e7f8f-dd68-4ca1-a0a2-191aaa3ae1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:23.790937 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e7f8f-dd68-4ca1-a0a2-191aaa3ae1ff None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:23.791425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e7f8f-dd68-4ca1-a0a2-191aaa3ae1ff None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:23.791899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e7f8f-dd68-4ca1-a0a2-191aaa3ae1ff None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:23.792812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e7f8f-dd68-4ca1-a0a2-191aaa3ae1ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:23.793337 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e7f8f-dd68-4ca1-a0a2-191aaa3ae1ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-f0bfdfc1-be79-4b68-8ffb-dda32964a9e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:23.794339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:23.795364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:23.796891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:23.796891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:23.796891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:23.796891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 307/597] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:23.796891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfdfc1-be79-4b68-8ffb-dda32964a9e0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:23.798742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e7f8f-dd68-4ca1-a0a2-191aaa3ae1ff None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:23.823248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 291/598] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:23.824438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e7f8f-dd68-4ca1-a0a2-191aaa3ae1ff None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:23.872053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:23.922426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bfdfc1-be79-4b68-8ffb-dda32964a9e0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:23.945339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:23.953580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e7f8f-dd68-4ca1-a0a2-191aaa3ae1ff None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:23.960590 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6be414-351f-40f4-9519-ae649883a75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:23.961191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6be414-351f-40f4-9519-ae649883a75d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:23.961465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6be414-351f-40f4-9519-ae649883a75d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:23.961733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6be414-351f-40f4-9519-ae649883a75d None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:23.962392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6be414-351f-40f4-9519-ae649883a75d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:23.962692 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6be414-351f-40f4-9519-ae649883a75d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:23.964089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6be414-351f-40f4-9519-ae649883a75d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:23.976243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 308/599] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:23.977951 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6be414-351f-40f4-9519-ae649883a75d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:23.989603 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192ebb8a-9d47-4a66-a0e3-f1a37f35020c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:23.990407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ebb8a-9d47-4a66-a0e3-f1a37f35020c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:23.990689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ebb8a-9d47-4a66-a0e3-f1a37f35020c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:23.990956 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ebb8a-9d47-4a66-a0e3-f1a37f35020c None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:23.991658 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ebb8a-9d47-4a66-a0e3-f1a37f35020c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:23.992055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ebb8a-9d47-4a66-a0e3-f1a37f35020c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:23.993399 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192ebb8a-9d47-4a66-a0e3-f1a37f35020c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:24.010146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 292/600] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:24.011363 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aff1e1-a48c-47f2-8649-0a1ef64b7d51 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:24.011652 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aff1e1-a48c-47f2-8649-0a1ef64b7d51 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:24.011924 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aff1e1-a48c-47f2-8649-0a1ef64b7d51 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:24.013930 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 293/601] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:24.018425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aff1e1-a48c-47f2-8649-0a1ef64b7d51 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:24.083443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6be414-351f-40f4-9519-ae649883a75d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:24.115592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:24.119531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2a06d-e879-446d-831c-49f38f1faab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:24.120198 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2a06d-e879-446d-831c-49f38f1faab2 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:24.120477 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2a06d-e879-446d-831c-49f38f1faab2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:24.120748 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2a06d-e879-446d-831c-49f38f1faab2 None admin] PATH_INFO: `/v3/projects/6bfc6c1c443849038af35577daf34088/users/4e021a00524341359bc4b5a800a9bbf6/roles/928c1b8875334543ab431e4f7e2acee5` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:24.125488 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aff1e1-a48c-47f2-8649-0a1ef64b7d51 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:24.135635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2a06d-e879-446d-831c-49f38f1faab2 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bfc6c1c443849038af35577daf34088, user_id=4e021a00524341359bc4b5a800a9bbf6, role_id=928c1b8875334543ab431e4f7e2acee5)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:24.137118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2a06d-e879-446d-831c-49f38f1faab2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:24.160189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae1667-0927-480f-963a-f2769a510002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:24.160815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae1667-0927-480f-963a-f2769a510002 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:24.161088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae1667-0927-480f-963a-f2769a510002 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:24.161354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bae1667-0927-480f-963a-f2769a510002 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:24.167760 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae1667-0927-480f-963a-f2769a510002 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:24.169269 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bae1667-0927-480f-963a-f2769a510002 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-8bae1667-0927-480f-963a-f2769a510002 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:24.180307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:24.181173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:24.182087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:24.182087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:24.182087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:24.182087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:24.182087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:24.182087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:24.182087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:24.182087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:24.182087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 294/602] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:24.182614 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae1667-0927-480f-963a-f2769a510002 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:24.190721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 309/603] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:34:23 2022] PUT /identity/v3/projects/6bfc6c1c443849038af35577daf34088/users/4e021a00524341359bc4b5a800a9bbf6/roles/928c1b8875334543ab431e4f7e2acee5 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:34:24.192237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee620c-e64d-429a-a7b2-f2c6cb8ea381 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:24.192947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee620c-e64d-429a-a7b2-f2c6cb8ea381 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:24.193220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee620c-e64d-429a-a7b2-f2c6cb8ea381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:24.214511 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-84ee620c-e64d-429a-a7b2-f2c6cb8ea381 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:24.216396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-84ee620c-e64d-429a-a7b2-f2c6cb8ea381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:24.245446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:24.300687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 310/604] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:24.301743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee620c-e64d-429a-a7b2-f2c6cb8ea381 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:24.342849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bae1667-0927-480f-963a-f2769a510002 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:24.418583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635ad89-2afa-4b50-9b9d-dbb4ca5677f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:24.420663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635ad89-2afa-4b50-9b9d-dbb4ca5677f3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:24.421088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635ad89-2afa-4b50-9b9d-dbb4ca5677f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:24.421735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9635ad89-2afa-4b50-9b9d-dbb4ca5677f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:24.435402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635ad89-2afa-4b50-9b9d-dbb4ca5677f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:24.437003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9635ad89-2afa-4b50-9b9d-dbb4ca5677f3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-9635ad89-2afa-4b50-9b9d-dbb4ca5677f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:24.450921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:24.452082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:24.453167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:24.453167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:24.453167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:24.453167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:24.453167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:24.453167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:24.453167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:24.453167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:24.454879 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 295/605] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:24.461060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635ad89-2afa-4b50-9b9d-dbb4ca5677f3 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:24.468571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee620c-e64d-429a-a7b2-f2c6cb8ea381 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:24.479611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:24.508251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aaee2e-dfaf-48f9-aa81-2ab448f97302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:24.509184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aaee2e-dfaf-48f9-aa81-2ab448f97302 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:24.509639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aaee2e-dfaf-48f9-aa81-2ab448f97302 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:24.510110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aaee2e-dfaf-48f9-aa81-2ab448f97302 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:24.510981 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aaee2e-dfaf-48f9-aa81-2ab448f97302 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:24.511481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aaee2e-dfaf-48f9-aa81-2ab448f97302 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:24.513701 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aaee2e-dfaf-48f9-aa81-2ab448f97302 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:24.526096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 311/606] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:24.532903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aaee2e-dfaf-48f9-aa81-2ab448f97302 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:24.555635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:24.595298 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9635ad89-2afa-4b50-9b9d-dbb4ca5677f3 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:24.640596 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aaee2e-dfaf-48f9-aa81-2ab448f97302 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:24.655495 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433c623-a7a0-41c2-b440-105d40b86538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:24.656449 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433c623-a7a0-41c2-b440-105d40b86538 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:24.656800 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433c623-a7a0-41c2-b440-105d40b86538 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:24.657196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9433c623-a7a0-41c2-b440-105d40b86538 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:24.658179 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433c623-a7a0-41c2-b440-105d40b86538 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:24.659384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433c623-a7a0-41c2-b440-105d40b86538 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:24.660827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9433c623-a7a0-41c2-b440-105d40b86538 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:24.673758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 296/607] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:24.674609 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433c623-a7a0-41c2-b440-105d40b86538 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:24.686440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7514e14-5aaf-49ab-a97a-4654dc4cb4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:24.687491 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7514e14-5aaf-49ab-a97a-4654dc4cb4c3 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:24.687988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7514e14-5aaf-49ab-a97a-4654dc4cb4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:24.688467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7514e14-5aaf-49ab-a97a-4654dc4cb4c3 None admin] PATH_INFO: `/v3/projects/6bfc6c1c443849038af35577daf34088/users/7103ff9544524393a8c7175998dcae3b/roles/928c1b8875334543ab431e4f7e2acee5` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:24.706093 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7514e14-5aaf-49ab-a97a-4654dc4cb4c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bfc6c1c443849038af35577daf34088, user_id=7103ff9544524393a8c7175998dcae3b, role_id=928c1b8875334543ab431e4f7e2acee5)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:24.708129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7514e14-5aaf-49ab-a97a-4654dc4cb4c3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:24.757735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:24.786086 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9433c623-a7a0-41c2-b440-105d40b86538 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:24.792933 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 312/608] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:34:24 2022] PUT /identity/v3/projects/6bfc6c1c443849038af35577daf34088/users/7103ff9544524393a8c7175998dcae3b/roles/928c1b8875334543ab431e4f7e2acee5 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:34:24.802884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7514e14-5aaf-49ab-a97a-4654dc4cb4c3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:24.825991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14987aff-d36b-4a61-990a-ba8189772deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:24.826991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14987aff-d36b-4a61-990a-ba8189772deb None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:24.827238 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14987aff-d36b-4a61-990a-ba8189772deb None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:24.827573 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14987aff-d36b-4a61-990a-ba8189772deb None admin] PATH_INFO: `/v3/roles/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:24.840817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14987aff-d36b-4a61-990a-ba8189772deb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-14987aff-d36b-4a61-990a-ba8189772deb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:24.846765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:24.848033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:24.849034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:24.849034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:24.849034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:24.849034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:24.849034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:24.849034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:24.849034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:24.849034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:24.849034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:24.849034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:24.849034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:24.849034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:24.850192 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 297/609] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:24.851681 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cc3cb-51d3-482e-aeb0-9aae0813f9fe None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:24.852067 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cc3cb-51d3-482e-aeb0-9aae0813f9fe None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:24.852426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655cc3cb-51d3-482e-aeb0-9aae0813f9fe None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:24.857850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 298/610] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:24.857850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cc3cb-51d3-482e-aeb0-9aae0813f9fe None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:24.903134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:24.941106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7514e14-5aaf-49ab-a97a-4654dc4cb4c3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:24.985694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1c390-e2db-40fa-97fa-01af25dae85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:24.986846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1c390-e2db-40fa-97fa-01af25dae85c None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:24.987281 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1c390-e2db-40fa-97fa-01af25dae85c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:24.988114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1c390-e2db-40fa-97fa-01af25dae85c None admin] PATH_INFO: `/v3/projects/6bfc6c1c443849038af35577daf34088/users/8a2c9eec08b142138569043e3b34c6a8/roles/928c1b8875334543ab431e4f7e2acee5` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:24.998289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655cc3cb-51d3-482e-aeb0-9aae0813f9fe None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:25.006232 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1c390-e2db-40fa-97fa-01af25dae85c None admin] RBAC: Authorizing `identity:create_grant(project_id=6bfc6c1c443849038af35577daf34088, user_id=8a2c9eec08b142138569043e3b34c6a8, role_id=928c1b8875334543ab431e4f7e2acee5)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:25.007988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1c390-e2db-40fa-97fa-01af25dae85c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:25.029286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:25.035619 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2d0d6-b8c5-44d8-a5e4-304305f67d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:25.036424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2d0d6-b8c5-44d8-a5e4-304305f67d93 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:25.036862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2d0d6-b8c5-44d8-a5e4-304305f67d93 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:25.037296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2d0d6-b8c5-44d8-a5e4-304305f67d93 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:25.038138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2d0d6-b8c5-44d8-a5e4-304305f67d93 None admin] RBAC: Adding query filter params (name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:25.038586 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2d0d6-b8c5-44d8-a5e4-304305f67d93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:25.040128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2d0d6-b8c5-44d8-a5e4-304305f67d93 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:25.048949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 299/611] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:25.050345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ebf2b-4424-4972-bac6-09e32bebab55 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:25.050631 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ebf2b-4424-4972-bac6-09e32bebab55 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:25.050903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ebf2b-4424-4972-bac6-09e32bebab55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:25.073326 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 313/612] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:34:24 2022] PUT /identity/v3/projects/6bfc6c1c443849038af35577daf34088/users/8a2c9eec08b142138569043e3b34c6a8/roles/928c1b8875334543ab431e4f7e2acee5 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:34:25.079259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1c390-e2db-40fa-97fa-01af25dae85c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:25.082209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-c40ebf2b-4424-4972-bac6-09e32bebab55 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:25.083466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c40ebf2b-4424-4972-bac6-09e32bebab55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:25.241651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 300/613] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:25.283555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1c390-e2db-40fa-97fa-01af25dae85c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:25.304260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:25.322935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e95cb3-ff2b-4629-80ec-f0e9780cc0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:25.324909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e95cb3-ff2b-4629-80ec-f0e9780cc0d3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:25.324909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e95cb3-ff2b-4629-80ec-f0e9780cc0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:25.324909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e95cb3-ff2b-4629-80ec-f0e9780cc0d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:25.335875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e95cb3-ff2b-4629-80ec-f0e9780cc0d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:25.337403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e95cb3-ff2b-4629-80ec-f0e9780cc0d3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-f6e95cb3-ff2b-4629-80ec-f0e9780cc0d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:25.353269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:25.359370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:25.360258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:25.360258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:25.360258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:25.360258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 314/614] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:25.362901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ebf2b-4424-4972-bac6-09e32bebab55 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:25.480655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ebf2b-4424-4972-bac6-09e32bebab55 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:25.517289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:25.535194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011f7b4-00d5-466f-b1cc-ca211e08d652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:25.535740 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011f7b4-00d5-466f-b1cc-ca211e08d652 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:25.536018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011f7b4-00d5-466f-b1cc-ca211e08d652 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:25.536319 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011f7b4-00d5-466f-b1cc-ca211e08d652 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:25.536967 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011f7b4-00d5-466f-b1cc-ca211e08d652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:25.537305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011f7b4-00d5-466f-b1cc-ca211e08d652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:25.538696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011f7b4-00d5-466f-b1cc-ca211e08d652 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:25.562136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 301/615] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:25.562924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e95cb3-ff2b-4629-80ec-f0e9780cc0d3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:25.585401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011f7b4-00d5-466f-b1cc-ca211e08d652 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:25.708410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011f7b4-00d5-466f-b1cc-ca211e08d652 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:25.748788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaf261-454f-4942-a019-935838839217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:25.749495 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaf261-454f-4942-a019-935838839217 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:25.749773 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaf261-454f-4942-a019-935838839217 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:25.750119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaf261-454f-4942-a019-935838839217 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:25.763863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccaf261-454f-4942-a019-935838839217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-3ccaf261-454f-4942-a019-935838839217 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:25.771489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:25.772497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:25.773334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:25.773334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:25.773334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:25.773334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:25.773334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:25.773334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:25.773334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:25.773334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:25.773334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:25.773334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:25.773334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:25.773334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:25.777706 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 302/616] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:25.779591 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaf261-454f-4942-a019-935838839217 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:25.819970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e95cb3-ff2b-4629-80ec-f0e9780cc0d3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:25.833170 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:25.887434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f883326-1718-4a8c-89b7-6afa1ae5f0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:25.888077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f883326-1718-4a8c-89b7-6afa1ae5f0ea None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:25.888365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f883326-1718-4a8c-89b7-6afa1ae5f0ea None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:25.890204 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f883326-1718-4a8c-89b7-6afa1ae5f0ea None admin] PATH_INFO: `/v3/users/swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:25.906618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f883326-1718-4a8c-89b7-6afa1ae5f0ea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:25.908078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f883326-1718-4a8c-89b7-6afa1ae5f0ea None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-0f883326-1718-4a8c-89b7-6afa1ae5f0ea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:25.929836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:25.930809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:25.931720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:25.931720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:25.931720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:25.931720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:25.931720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:25.931720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:25.931720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:25.931720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:25.931720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 19:34:25.931720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:25.932411 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaf261-454f-4942-a019-935838839217 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:25.934129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 315/617] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:34:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:25.940274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f883326-1718-4a8c-89b7-6afa1ae5f0ea None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:25.994964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5f4dc-a9be-46f6-b9ad-9d3c9527e21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:25.996301 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5f4dc-a9be-46f6-b9ad-9d3c9527e21e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:25.996584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5f4dc-a9be-46f6-b9ad-9d3c9527e21e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:25.997281 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:25.997440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5f4dc-a9be-46f6-b9ad-9d3c9527e21e None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:25.998047 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5f4dc-a9be-46f6-b9ad-9d3c9527e21e None admin] RBAC: Adding query filter params (name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:25.998344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5f4dc-a9be-46f6-b9ad-9d3c9527e21e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:26.000560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5f4dc-a9be-46f6-b9ad-9d3c9527e21e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:26.025928 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 303/618] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:26.028010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5f4dc-a9be-46f6-b9ad-9d3c9527e21e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:26.121471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f883326-1718-4a8c-89b7-6afa1ae5f0ea None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:26.188813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd5d74-577c-4c68-814c-20ac90536f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:26.189431 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd5d74-577c-4c68-814c-20ac90536f2a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:26.189712 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd5d74-577c-4c68-814c-20ac90536f2a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:26.192720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd5d74-577c-4c68-814c-20ac90536f2a None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:26.193464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd5d74-577c-4c68-814c-20ac90536f2a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:26.193774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd5d74-577c-4c68-814c-20ac90536f2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:26.195244 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd5d74-577c-4c68-814c-20ac90536f2a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:26.225738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5f4dc-a9be-46f6-b9ad-9d3c9527e21e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:26.231950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 316/619] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Sep 14 19:34:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:26.238097 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd5d74-577c-4c68-814c-20ac90536f2a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:26.241007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:26.292507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a393e-b3ca-4f49-8470-a7d6e72d5698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:26.293140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a393e-b3ca-4f49-8470-a7d6e72d5698 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:26.293417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a393e-b3ca-4f49-8470-a7d6e72d5698 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:26.294234 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a393e-b3ca-4f49-8470-a7d6e72d5698 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:26.315568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a393e-b3ca-4f49-8470-a7d6e72d5698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:26.317020 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a393e-b3ca-4f49-8470-a7d6e72d5698 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-d84a393e-b3ca-4f49-8470-a7d6e72d5698 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:26.344191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:26.346441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:26.348519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:26.348519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:26.348519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:26.351175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 304/620] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:26.354978 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a393e-b3ca-4f49-8470-a7d6e72d5698 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:26.394674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:26.406121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd5d74-577c-4c68-814c-20ac90536f2a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:26.445315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c264b-7b20-42b6-8163-a8bc9a12fb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:26.446348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c264b-7b20-42b6-8163-a8bc9a12fb32 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:26.446841 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c264b-7b20-42b6-8163-a8bc9a12fb32 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:26.447333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292c264b-7b20-42b6-8163-a8bc9a12fb32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:26.458324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c264b-7b20-42b6-8163-a8bc9a12fb32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:26.460216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292c264b-7b20-42b6-8163-a8bc9a12fb32 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-292c264b-7b20-42b6-8163-a8bc9a12fb32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:26.477651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:26.478631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:26.479509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:26.479509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:26.479509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:26.482100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 317/621] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:26.495909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c264b-7b20-42b6-8163-a8bc9a12fb32 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:26.576701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a393e-b3ca-4f49-8470-a7d6e72d5698 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:26.635030 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77466ec8-2e24-4fbc-b6a3-80b05ddd38cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:26.635997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77466ec8-2e24-4fbc-b6a3-80b05ddd38cd None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:26.636512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77466ec8-2e24-4fbc-b6a3-80b05ddd38cd None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:26.637146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77466ec8-2e24-4fbc-b6a3-80b05ddd38cd None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:26.638123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77466ec8-2e24-4fbc-b6a3-80b05ddd38cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:26.638639 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77466ec8-2e24-4fbc-b6a3-80b05ddd38cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:26.640755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77466ec8-2e24-4fbc-b6a3-80b05ddd38cd None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:26.662289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 305/622] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:26.666670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77466ec8-2e24-4fbc-b6a3-80b05ddd38cd None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:26.699070 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:26.700510 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292c264b-7b20-42b6-8163-a8bc9a12fb32 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:26.757736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dac80-0c11-41b7-8476-d854fce9f782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:26.758407 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dac80-0c11-41b7-8476-d854fce9f782 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:26.758686 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dac80-0c11-41b7-8476-d854fce9f782 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:26.758954 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7dac80-0c11-41b7-8476-d854fce9f782 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:26.759591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dac80-0c11-41b7-8476-d854fce9f782 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:26.759888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dac80-0c11-41b7-8476-d854fce9f782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:26.761428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7dac80-0c11-41b7-8476-d854fce9f782 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:26.775563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 318/623] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:26.781555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dac80-0c11-41b7-8476-d854fce9f782 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:26.784531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:26.880048 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77466ec8-2e24-4fbc-b6a3-80b05ddd38cd None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:26.901189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7dac80-0c11-41b7-8476-d854fce9f782 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:26.921216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476711b-b924-4c1e-9f59-8e5fa019fca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:26.921786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476711b-b924-4c1e-9f59-8e5fa019fca8 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:26.922115 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476711b-b924-4c1e-9f59-8e5fa019fca8 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:26.922883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476711b-b924-4c1e-9f59-8e5fa019fca8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:26.933106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476711b-b924-4c1e-9f59-8e5fa019fca8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:26.935219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476711b-b924-4c1e-9f59-8e5fa019fca8 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:26.936740 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6668a-bb2f-4007-b710-b63c448d6827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:26.937517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6668a-bb2f-4007-b710-b63c448d6827 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:26.937945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6668a-bb2f-4007-b710-b63c448d6827 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:26.938298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf6668a-bb2f-4007-b710-b63c448d6827 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:26.946025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6668a-bb2f-4007-b710-b63c448d6827 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:26.947600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf6668a-bb2f-4007-b710-b63c448d6827 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-b476711b-b924-4c1e-9f59-8e5fa019fca8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:26.950844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:26.952013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:26.953080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:26.953080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:26.953080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:26.953080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:26.953080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:26.953080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:26.953080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:26.953080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:26.953080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:34:26.953080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:26.953711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 306/624] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 19:34:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:26.954833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da4bbd-6acd-4943-9eed-ab9d90d64f4e None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:26.957839 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da4bbd-6acd-4943-9eed-ab9d90d64f4e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:26.957839 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da4bbd-6acd-4943-9eed-ab9d90d64f4e None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:26.957839 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 307/625] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:26.965195 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3283d-25a5-4e23-80d3-fa4a90161ea8 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:26.965893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3283d-25a5-4e23-80d3-fa4a90161ea8 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:26.966362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff3283d-25a5-4e23-80d3-fa4a90161ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-daf6668a-bb2f-4007-b710-b63c448d6827 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:26.969245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:26.970237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:26.971114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:26.971114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:26.971114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:26.971114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:26.971114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:26.971114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:26.971114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:26.971114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:26.971114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 319/626] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:26.972430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6668a-bb2f-4007-b710-b63c448d6827 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:26.990283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-bff3283d-25a5-4e23-80d3-fa4a90161ea8 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:26.991760 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bff3283d-25a5-4e23-80d3-fa4a90161ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:27.043065 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:27.070065 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 308/627] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:27.071290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3283d-25a5-4e23-80d3-fa4a90161ea8 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:27.082412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:27.088839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf6668a-bb2f-4007-b710-b63c448d6827 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:27.128510 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6479814-f6e4-4cd5-8626-5036628e244d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:27.129580 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6479814-f6e4-4cd5-8626-5036628e244d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:27.129946 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6479814-f6e4-4cd5-8626-5036628e244d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:27.130348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6479814-f6e4-4cd5-8626-5036628e244d None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:27.131586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6479814-f6e4-4cd5-8626-5036628e244d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:27.132202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6479814-f6e4-4cd5-8626-5036628e244d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:27.133836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6479814-f6e4-4cd5-8626-5036628e244d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:27.156139 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 320/628] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 19:34:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:27.160763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d3ed1-831a-48e1-8864-876650690989 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:27.160763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d3ed1-831a-48e1-8864-876650690989 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:27.160763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d3ed1-831a-48e1-8864-876650690989 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:27.160763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 321/629] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:27.161759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d3ed1-831a-48e1-8864-876650690989 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:27.183333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff3283d-25a5-4e23-80d3-fa4a90161ea8 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:27.219832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967499e6-69da-4407-b6f7-dda7c3c99f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:27.220748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967499e6-69da-4407-b6f7-dda7c3c99f60 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:27.221209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967499e6-69da-4407-b6f7-dda7c3c99f60 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:27.221708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967499e6-69da-4407-b6f7-dda7c3c99f60 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:27.222653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967499e6-69da-4407-b6f7-dda7c3c99f60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:27.223167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967499e6-69da-4407-b6f7-dda7c3c99f60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:27.224830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967499e6-69da-4407-b6f7-dda7c3c99f60 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:27.237525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 309/630] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:27.239113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d710d-1c7e-4faa-a2d1-8d4d937dd13f None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:27.239622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d710d-1c7e-4faa-a2d1-8d4d937dd13f None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:27.240098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d710d-1c7e-4faa-a2d1-8d4d937dd13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:27.259084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e58d710d-1c7e-4faa-a2d1-8d4d937dd13f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:27.260585 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e58d710d-1c7e-4faa-a2d1-8d4d937dd13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:27.285947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d3ed1-831a-48e1-8864-876650690989 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:27.312161 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:27.329836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbb01b-c763-4ee6-8a62-781365ac4696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:27.331721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbb01b-c763-4ee6-8a62-781365ac4696 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:27.331721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbb01b-c763-4ee6-8a62-781365ac4696 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:27.331721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbb01b-c763-4ee6-8a62-781365ac4696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:27.336948 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:27.343943 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 310/631] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:27.345239 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d710d-1c7e-4faa-a2d1-8d4d937dd13f None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:27.414463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbb01b-c763-4ee6-8a62-781365ac4696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:27.416154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbb01b-c763-4ee6-8a62-781365ac4696 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:27.468981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58d710d-1c7e-4faa-a2d1-8d4d937dd13f None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:27.516573 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f448e83-3631-4663-9a0e-5a36447e8205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:27.517242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f448e83-3631-4663-9a0e-5a36447e8205 None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:27.517524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f448e83-3631-4663-9a0e-5a36447e8205 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:27.517827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f448e83-3631-4663-9a0e-5a36447e8205 None admin] PATH_INFO: `/v3/projects/6bfc6c1c443849038af35577daf34088/users/e3d2f1ab1bfd435183e33168400c0b17/roles/928c1b8875334543ab431e4f7e2acee5` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:27.537847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f448e83-3631-4663-9a0e-5a36447e8205 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bfc6c1c443849038af35577daf34088, user_id=e3d2f1ab1bfd435183e33168400c0b17, role_id=928c1b8875334543ab431e4f7e2acee5)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:27.538421 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 322/632] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:34:27.538598 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f448e83-3631-4663-9a0e-5a36447e8205 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:27.539713 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbb01b-c763-4ee6-8a62-781365ac4696 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:27.562930 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:27.621835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 311/633] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:34:27 2022] PUT /identity/v3/projects/6bfc6c1c443849038af35577daf34088/users/e3d2f1ab1bfd435183e33168400c0b17/roles/928c1b8875334543ab431e4f7e2acee5 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:34:27.621835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f448e83-3631-4663-9a0e-5a36447e8205 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:27.655356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbb01b-c763-4ee6-8a62-781365ac4696 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:27.694321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:27.697441 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305178e-833b-4d27-b78f-655498cd0c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:27.698141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305178e-833b-4d27-b78f-655498cd0c8f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:27.698418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305178e-833b-4d27-b78f-655498cd0c8f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:27.698691 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3305178e-833b-4d27-b78f-655498cd0c8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:27.714257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3305178e-833b-4d27-b78f-655498cd0c8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-3305178e-833b-4d27-b78f-655498cd0c8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:27.719581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:27.720532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:27.726671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:27.726671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:27.726671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:27.726671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:27.726671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:27.726671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:27.726671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:27.726671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:27.726671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:27.726671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:27.726671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:27.726671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:27.726671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 323/634] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:27.726671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305178e-833b-4d27-b78f-655498cd0c8f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:27.773160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f448e83-3631-4663-9a0e-5a36447e8205 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:27.820462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:27.821207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726e2e9-67e6-4769-b809-976205b6ca4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:27.822396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726e2e9-67e6-4769-b809-976205b6ca4f None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:27.822396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726e2e9-67e6-4769-b809-976205b6ca4f None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:27.822564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c726e2e9-67e6-4769-b809-976205b6ca4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:27.834602 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726e2e9-67e6-4769-b809-976205b6ca4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:27.836054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c726e2e9-67e6-4769-b809-976205b6ca4f None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:27.843731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3305178e-833b-4d27-b78f-655498cd0c8f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-c726e2e9-67e6-4769-b809-976205b6ca4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:27.851103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:27.852021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:27.852917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:27.852917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:27.852917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:27.852917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 312/635] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:27.853138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726e2e9-67e6-4769-b809-976205b6ca4f None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:27.889573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba397c-d903-49c7-8fca-0b93ed70ad4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:27.890245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba397c-d903-49c7-8fca-0b93ed70ad4c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:27.890523 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba397c-d903-49c7-8fca-0b93ed70ad4c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:27.890793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba397c-d903-49c7-8fca-0b93ed70ad4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:27.901887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ba397c-d903-49c7-8fca-0b93ed70ad4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-f0ba397c-d903-49c7-8fca-0b93ed70ad4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:27.907020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:27.907993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:27.908836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:27.908836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:27.908836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:27.908836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:27.908836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:27.908836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:27.908836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:27.908836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:27.908836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:27.908836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:27.908836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:27.908836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:27.908836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 324/636] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:27.909520 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3842cd4-f701-4af6-8392-76ed5fbf63f3 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:27.910289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3842cd4-f701-4af6-8392-76ed5fbf63f3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:27.910289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3842cd4-f701-4af6-8392-76ed5fbf63f3 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:27.913923 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 325/637] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:27.913923 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3842cd4-f701-4af6-8392-76ed5fbf63f3 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:27.962189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:27.965182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c726e2e9-67e6-4769-b809-976205b6ca4f None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:28.002583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddfaf5-a4ff-4e5d-be32-dc8811d2fa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:28.003145 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddfaf5-a4ff-4e5d-be32-dc8811d2fa81 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:28.003428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddfaf5-a4ff-4e5d-be32-dc8811d2fa81 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:28.003699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ddfaf5-a4ff-4e5d-be32-dc8811d2fa81 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:28.004384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddfaf5-a4ff-4e5d-be32-dc8811d2fa81 None admin] RBAC: Adding query filter params (name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:28.004690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddfaf5-a4ff-4e5d-be32-dc8811d2fa81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:28.006080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ddfaf5-a4ff-4e5d-be32-dc8811d2fa81 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:28.014161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 313/638] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:28.015136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddfaf5-a4ff-4e5d-be32-dc8811d2fa81 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:28.030987 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3842cd4-f701-4af6-8392-76ed5fbf63f3 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:28.066802 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be9d9dd-fed8-4402-864c-c002f0d190ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:28.067656 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9d9dd-fed8-4402-864c-c002f0d190ab None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:28.067926 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9d9dd-fed8-4402-864c-c002f0d190ab None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:28.068188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be9d9dd-fed8-4402-864c-c002f0d190ab None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:28.068872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9d9dd-fed8-4402-864c-c002f0d190ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:28.069166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9d9dd-fed8-4402-864c-c002f0d190ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:28.070540 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be9d9dd-fed8-4402-864c-c002f0d190ab None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:28.079598 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:28.087335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 326/639] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:28.087335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a693c07-73e3-4e8d-8fc9-1dc4906c95b7 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:28.087335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a693c07-73e3-4e8d-8fc9-1dc4906c95b7 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:28.087335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a693c07-73e3-4e8d-8fc9-1dc4906c95b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:28.105125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3a693c07-73e3-4e8d-8fc9-1dc4906c95b7 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:28.106942 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3a693c07-73e3-4e8d-8fc9-1dc4906c95b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:28.120517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ddfaf5-a4ff-4e5d-be32-dc8811d2fa81 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:28.157673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67d8b1-4528-4c58-bb28-76751d78c0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:28.158317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67d8b1-4528-4c58-bb28-76751d78c0af None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:28.158592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67d8b1-4528-4c58-bb28-76751d78c0af None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:28.158861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67d8b1-4528-4c58-bb28-76751d78c0af None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:28.159568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67d8b1-4528-4c58-bb28-76751d78c0af None admin] RBAC: Adding query filter params (name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:28.159871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67d8b1-4528-4c58-bb28-76751d78c0af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:28.161220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67d8b1-4528-4c58-bb28-76751d78c0af None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:28.175009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 314/640] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:28.175009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67d8b1-4528-4c58-bb28-76751d78c0af None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:28.185173 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 327/641] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:28.194816 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a693c07-73e3-4e8d-8fc9-1dc4906c95b7 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:28.203559 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:28.285658 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67d8b1-4528-4c58-bb28-76751d78c0af None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:28.317716 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a693c07-73e3-4e8d-8fc9-1dc4906c95b7 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:28.319949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8306d7-b058-4eeb-b725-4b3062895f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:28.320668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8306d7-b058-4eeb-b725-4b3062895f09 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:28.321015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8306d7-b058-4eeb-b725-4b3062895f09 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:28.321369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8306d7-b058-4eeb-b725-4b3062895f09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:28.332652 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8306d7-b058-4eeb-b725-4b3062895f09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:28.334223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8306d7-b058-4eeb-b725-4b3062895f09 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-0d8306d7-b058-4eeb-b725-4b3062895f09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:28.348561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:28.349513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:28.350627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:28.350627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:28.350627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:28.350627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 315/642] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:28.351621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8306d7-b058-4eeb-b725-4b3062895f09 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:28.355636 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:28.362843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d11a1-8b91-4a05-99bf-941df6d672c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:28.363722 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d11a1-8b91-4a05-99bf-941df6d672c6 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:28.364195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d11a1-8b91-4a05-99bf-941df6d672c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:28.364607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d11a1-8b91-4a05-99bf-941df6d672c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:28.373253 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d11a1-8b91-4a05-99bf-941df6d672c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:28.375046 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d11a1-8b91-4a05-99bf-941df6d672c6 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-202d11a1-8b91-4a05-99bf-941df6d672c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:28.389948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:28.390974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:28.391889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:28.391889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:28.391889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:28.391889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:28.391889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:28.391889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:28.391889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:28.391889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:28.398769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 328/643] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:28.399817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d11a1-8b91-4a05-99bf-941df6d672c6 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:28.437096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:28.461504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8306d7-b058-4eeb-b725-4b3062895f09 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:28.499891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabf55e-179d-445e-a518-4d715bd9adbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:28.500518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf55e-179d-445e-a518-4d715bd9adbf None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:28.500794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf55e-179d-445e-a518-4d715bd9adbf None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:28.501062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf55e-179d-445e-a518-4d715bd9adbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:28.512088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabf55e-179d-445e-a518-4d715bd9adbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:28.513627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabf55e-179d-445e-a518-4d715bd9adbf None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:28.514354 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d11a1-8b91-4a05-99bf-941df6d672c6 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-0fabf55e-179d-445e-a518-4d715bd9adbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:28.528692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:28.529993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:28.531220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:28.531220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:28.531220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:28.531220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 316/644] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:28.531220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabf55e-179d-445e-a518-4d715bd9adbf None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:28.559693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dfa3e-2cac-4672-982d-2a7dfde2405f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:28.560200 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dfa3e-2cac-4672-982d-2a7dfde2405f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:28.560472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dfa3e-2cac-4672-982d-2a7dfde2405f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:28.560740 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dfa3e-2cac-4672-982d-2a7dfde2405f None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:28.561479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dfa3e-2cac-4672-982d-2a7dfde2405f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:28.561778 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dfa3e-2cac-4672-982d-2a7dfde2405f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:28.563245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dfa3e-2cac-4672-982d-2a7dfde2405f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:28.576427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 329/645] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:28.577305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dfa3e-2cac-4672-982d-2a7dfde2405f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:28.633389 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:28.641105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabf55e-179d-445e-a518-4d715bd9adbf None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:28.666634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:28.674824 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01723403-dc06-444c-ab37-4eac38f96c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:28.675510 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01723403-dc06-444c-ab37-4eac38f96c8b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:28.675907 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01723403-dc06-444c-ab37-4eac38f96c8b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:28.676183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01723403-dc06-444c-ab37-4eac38f96c8b None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:28.676863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01723403-dc06-444c-ab37-4eac38f96c8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:28.677153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01723403-dc06-444c-ab37-4eac38f96c8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:28.678670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01723403-dc06-444c-ab37-4eac38f96c8b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:28.690234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dfa3e-2cac-4672-982d-2a7dfde2405f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:28.690901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 317/646] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:28.691681 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01723403-dc06-444c-ab37-4eac38f96c8b None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:28.729021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3dde9-81da-4969-8127-8e979a5bc513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:28.729725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3dde9-81da-4969-8127-8e979a5bc513 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:28.730051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3dde9-81da-4969-8127-8e979a5bc513 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:28.730322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3dde9-81da-4969-8127-8e979a5bc513 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:28.740529 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3dde9-81da-4969-8127-8e979a5bc513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-b1a3dde9-81da-4969-8127-8e979a5bc513 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:28.745193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:28.746129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:28.748271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:28.748271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:28.748271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:28.748271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:28.748271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:28.748271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:28.748271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:28.748271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:28.748271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:28.748271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:28.748271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:28.748271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:28.748271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 330/647] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:28.748271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3dde9-81da-4969-8127-8e979a5bc513 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:28.803135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01723403-dc06-444c-ab37-4eac38f96c8b None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:28.839928 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d6b12-e59c-41de-b6dc-4d389521c73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:28.841028 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d6b12-e59c-41de-b6dc-4d389521c73e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:28.841305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d6b12-e59c-41de-b6dc-4d389521c73e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:28.841574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d6b12-e59c-41de-b6dc-4d389521c73e None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:28.842235 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d6b12-e59c-41de-b6dc-4d389521c73e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:28.842537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d6b12-e59c-41de-b6dc-4d389521c73e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:28.843931 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d6b12-e59c-41de-b6dc-4d389521c73e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:28.855489 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3dde9-81da-4969-8127-8e979a5bc513 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:28.856410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 318/648] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:28.857395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d6b12-e59c-41de-b6dc-4d389521c73e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:28.871402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:28.895227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70c455-aefd-435a-9c90-b404027bc795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:28.895973 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70c455-aefd-435a-9c90-b404027bc795 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:28.896245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70c455-aefd-435a-9c90-b404027bc795 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:28.896507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70c455-aefd-435a-9c90-b404027bc795 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:28.903957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70c455-aefd-435a-9c90-b404027bc795 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:28.905463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd70c455-aefd-435a-9c90-b404027bc795 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:28.913827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-cd70c455-aefd-435a-9c90-b404027bc795 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:28.919526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:28.922800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:28.923725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:28.923725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:28.923725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:28.923725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:28.923725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:28.923725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:28.923725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:28.923725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:28.923725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 19:34:28.923725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:28.923725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 331/649] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:28.923725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70c455-aefd-435a-9c90-b404027bc795 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:28.974372 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d6b12-e59c-41de-b6dc-4d389521c73e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:29.007010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6c550-2f2b-4fd7-8057-11a81bb5599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:29.007595 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6c550-2f2b-4fd7-8057-11a81bb5599e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:29.007870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6c550-2f2b-4fd7-8057-11a81bb5599e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:29.008137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf6c550-2f2b-4fd7-8057-11a81bb5599e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:29.008846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6c550-2f2b-4fd7-8057-11a81bb5599e None admin] RBAC: Adding query filter params (role.id=928c1b8875334543ab431e4f7e2acee5, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=2a4284b59f804dc59674d621d87634bc) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:29.009313 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6c550-2f2b-4fd7-8057-11a81bb5599e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:29.010747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf6c550-2f2b-4fd7-8057-11a81bb5599e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:29.025832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 319/650] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:28 2022] GET /identity/v3/role_assignments?user.id=2a4284b59f804dc59674d621d87634bc&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=928c1b8875334543ab431e4f7e2acee5 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:29.025832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6c550-2f2b-4fd7-8057-11a81bb5599e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:29.047246 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd70c455-aefd-435a-9c90-b404027bc795 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:29.096012 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca92af-5c39-4b41-bb09-9bd84727ac06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:29.096701 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca92af-5c39-4b41-bb09-9bd84727ac06 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:29.097210 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca92af-5c39-4b41-bb09-9bd84727ac06 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:29.097480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aca92af-5c39-4b41-bb09-9bd84727ac06 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:29.098602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca92af-5c39-4b41-bb09-9bd84727ac06 None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:29.098904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca92af-5c39-4b41-bb09-9bd84727ac06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:29.100291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aca92af-5c39-4b41-bb09-9bd84727ac06 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:29.119212 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 332/651] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:29.120239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca92af-5c39-4b41-bb09-9bd84727ac06 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:29.158721 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf6c550-2f2b-4fd7-8057-11a81bb5599e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:29.170322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:29.178970 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:29.207651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812ee2-7b9b-4952-ad4e-559ee70824b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:29.208626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812ee2-7b9b-4952-ad4e-559ee70824b6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:29.209539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812ee2-7b9b-4952-ad4e-559ee70824b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:29.210392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10812ee2-7b9b-4952-ad4e-559ee70824b6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:29.219138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10812ee2-7b9b-4952-ad4e-559ee70824b6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:29.220583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10812ee2-7b9b-4952-ad4e-559ee70824b6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-10812ee2-7b9b-4952-ad4e-559ee70824b6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:29.232919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:29.233935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:29.235221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:29.235221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:29.235221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:29.235221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:29.235221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:29.235221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:29.235221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:29.235221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:29.235221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 19:34:29.235221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:29.235221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 320/652] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:29.235221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812ee2-7b9b-4952-ad4e-559ee70824b6 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:29.250287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aca92af-5c39-4b41-bb09-9bd84727ac06 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:29.295218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e9e8b-5aca-4be7-992b-a6bac3b4c42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:29.295823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e9e8b-5aca-4be7-992b-a6bac3b4c42a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:29.296097 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e9e8b-5aca-4be7-992b-a6bac3b4c42a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:29.296363 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e9e8b-5aca-4be7-992b-a6bac3b4c42a None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:29.297078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e9e8b-5aca-4be7-992b-a6bac3b4c42a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:29.297379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e9e8b-5aca-4be7-992b-a6bac3b4c42a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:29.298837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58e9e8b-5aca-4be7-992b-a6bac3b4c42a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:29.320028 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 333/653] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Sep 14 19:34:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:29.320830 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e9e8b-5aca-4be7-992b-a6bac3b4c42a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:29.366344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10812ee2-7b9b-4952-ad4e-559ee70824b6 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:29.403442 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec538ec-36a8-4c1a-bca6-1f77103c1005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:29.404075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec538ec-36a8-4c1a-bca6-1f77103c1005 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:29.404351 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec538ec-36a8-4c1a-bca6-1f77103c1005 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:29.404621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec538ec-36a8-4c1a-bca6-1f77103c1005 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:29.415500 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec538ec-36a8-4c1a-bca6-1f77103c1005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:29.416959 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec538ec-36a8-4c1a-bca6-1f77103c1005 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-bec538ec-36a8-4c1a-bca6-1f77103c1005 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:29.432111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:29.433106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:29.434001 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:29.434001 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:29.434001 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:29.434001 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 321/654] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:29.434814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec538ec-36a8-4c1a-bca6-1f77103c1005 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:29.439739 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:29.451572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e9e8b-5aca-4be7-992b-a6bac3b4c42a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:29.463247 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:29.498538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0475d07-0f4c-4fc5-b206-cc898f888d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:29.499422 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0475d07-0f4c-4fc5-b206-cc898f888d79 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:29.499422 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0475d07-0f4c-4fc5-b206-cc898f888d79 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:29.499712 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0475d07-0f4c-4fc5-b206-cc898f888d79 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:29.500433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0475d07-0f4c-4fc5-b206-cc898f888d79 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:29.500733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0475d07-0f4c-4fc5-b206-cc898f888d79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:29.502179 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0475d07-0f4c-4fc5-b206-cc898f888d79 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:29.525692 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 334/655] 10.222.0.42 () {56 vars in 1139 bytes} [Wed Sep 14 19:34:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:29.526670 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0475d07-0f4c-4fc5-b206-cc898f888d79 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:29.563370 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec538ec-36a8-4c1a-bca6-1f77103c1005 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:29.606519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f5688-284c-4d89-8945-c7f35e50fd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:29.607393 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f5688-284c-4d89-8945-c7f35e50fd4c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:29.607815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f5688-284c-4d89-8945-c7f35e50fd4c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:29.608227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f5688-284c-4d89-8945-c7f35e50fd4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:29.634441 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0475d07-0f4c-4fc5-b206-cc898f888d79 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:29.671491 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9b544-24c0-4949-8e37-9006f6f9b84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:29.672128 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9b544-24c0-4949-8e37-9006f6f9b84b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:29.672604 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9b544-24c0-4949-8e37-9006f6f9b84b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:29.672720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce9b544-24c0-4949-8e37-9006f6f9b84b None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:29.673376 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9b544-24c0-4949-8e37-9006f6f9b84b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:29.673711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9b544-24c0-4949-8e37-9006f6f9b84b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:29.675137 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce9b544-24c0-4949-8e37-9006f6f9b84b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:29.679825 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f5688-284c-4d89-8945-c7f35e50fd4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:29.681770 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212f5688-284c-4d89-8945-c7f35e50fd4c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:29.692061 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 335/656] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:29.692061 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9b544-24c0-4949-8e37-9006f6f9b84b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:29.701838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:29.726233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:29.799346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 322/657] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:34:29.800358 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f5688-284c-4d89-8945-c7f35e50fd4c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:29.803201 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce9b544-24c0-4949-8e37-9006f6f9b84b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:29.844117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a866a64e-cdea-49bb-b583-3dad4cbda69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:29.845154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866a64e-cdea-49bb-b583-3dad4cbda69b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:29.845432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866a64e-cdea-49bb-b583-3dad4cbda69b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:29.845699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a866a64e-cdea-49bb-b583-3dad4cbda69b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:29.911344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212f5688-284c-4d89-8945-c7f35e50fd4c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:29.925436 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866a64e-cdea-49bb-b583-3dad4cbda69b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:29.926936 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a866a64e-cdea-49bb-b583-3dad4cbda69b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:29.935917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:29.936930 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:29.947112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bdd22-0d17-480e-90c4-7f3e6cd1f190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:29.947949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bdd22-0d17-480e-90c4-7f3e6cd1f190 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:29.948060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bdd22-0d17-480e-90c4-7f3e6cd1f190 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:29.948254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902bdd22-0d17-480e-90c4-7f3e6cd1f190 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:29.955688 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902bdd22-0d17-480e-90c4-7f3e6cd1f190 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:29.957107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902bdd22-0d17-480e-90c4-7f3e6cd1f190 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-902bdd22-0d17-480e-90c4-7f3e6cd1f190 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:29.969504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:29.970674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:29.971641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:29.971641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:29.971641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:29.971641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:29.971641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:29.971641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:29.971641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:29.971641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:29.971641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 19:34:29.971641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:29.971641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 323/658] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:29.971641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bdd22-0d17-480e-90c4-7f3e6cd1f190 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:30.042539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 336/659] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:34:30.042539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001ee52-d222-446e-bea7-4323bf2fc785 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:30.042539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001ee52-d222-446e-bea7-4323bf2fc785 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:30.042539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3001ee52-d222-446e-bea7-4323bf2fc785 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:30.045891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 337/660] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:30.046671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001ee52-d222-446e-bea7-4323bf2fc785 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:30.087568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902bdd22-0d17-480e-90c4-7f3e6cd1f190 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:30.123885 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2edcd2-393e-4abd-ba35-454694a2a480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:30.124516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2edcd2-393e-4abd-ba35-454694a2a480 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:30.124791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2edcd2-393e-4abd-ba35-454694a2a480 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:30.125063 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2edcd2-393e-4abd-ba35-454694a2a480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:30.139204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2edcd2-393e-4abd-ba35-454694a2a480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:30.141026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2edcd2-393e-4abd-ba35-454694a2a480 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:30.157384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3001ee52-d222-446e-bea7-4323bf2fc785 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-4e2edcd2-393e-4abd-ba35-454694a2a480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:30.157858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:30.158771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:30.159657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:30.159657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:30.159657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:30.159657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 324/661] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:30.159893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2edcd2-393e-4abd-ba35-454694a2a480 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:30.185424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:30.198329 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:30.206434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b38f2-3a80-4fa7-9d84-e2cdf0a7e564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:30.206434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b38f2-3a80-4fa7-9d84-e2cdf0a7e564 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:30.206434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b38f2-3a80-4fa7-9d84-e2cdf0a7e564 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:30.206434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b38f2-3a80-4fa7-9d84-e2cdf0a7e564 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:30.206815 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b38f2-3a80-4fa7-9d84-e2cdf0a7e564 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:30.206815 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b38f2-3a80-4fa7-9d84-e2cdf0a7e564 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:30.206815 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b38f2-3a80-4fa7-9d84-e2cdf0a7e564 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:30.226433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 338/662] 10.222.0.42 () {56 vars in 1137 bytes} [Wed Sep 14 19:34:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:30.226433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e183801-ae0f-4ff8-ac42-ff59e0d6c9b1 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:30.226433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e183801-ae0f-4ff8-ac42-ff59e0d6c9b1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:30.226433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e183801-ae0f-4ff8-ac42-ff59e0d6c9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:30.243630 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1e183801-ae0f-4ff8-ac42-ff59e0d6c9b1 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:30.244825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1e183801-ae0f-4ff8-ac42-ff59e0d6c9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:30.270913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2edcd2-393e-4abd-ba35-454694a2a480 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:30.305758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb9286-99dd-4b1e-bc1e-fb3cc26720bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:30.306614 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb9286-99dd-4b1e-bc1e-fb3cc26720bd None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:30.306809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb9286-99dd-4b1e-bc1e-fb3cc26720bd None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:30.307079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb9286-99dd-4b1e-bc1e-fb3cc26720bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:30.325985 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 339/663] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:30.325985 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e183801-ae0f-4ff8-ac42-ff59e0d6c9b1 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:30.326219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb9286-99dd-4b1e-bc1e-fb3cc26720bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:30.326219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bb9286-99dd-4b1e-bc1e-fb3cc26720bd None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-31bb9286-99dd-4b1e-bc1e-fb3cc26720bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:30.334719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:30.335724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:30.336607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:30.336607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:30.336607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:30.336607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 325/664] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:30.336844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb9286-99dd-4b1e-bc1e-fb3cc26720bd None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:30.440171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:30.456109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb9286-99dd-4b1e-bc1e-fb3cc26720bd None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:30.457947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e183801-ae0f-4ff8-ac42-ff59e0d6c9b1 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:30.466231 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:30.503102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed6a2f-b5a6-4eff-af8a-61dc7d8dded5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:30.503694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed6a2f-b5a6-4eff-af8a-61dc7d8dded5 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:30.503968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed6a2f-b5a6-4eff-af8a-61dc7d8dded5 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:30.504342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbed6a2f-b5a6-4eff-af8a-61dc7d8dded5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:30.509403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d00b16-a6aa-4b68-ae12-b32cb0c18089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:30.510419 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d00b16-a6aa-4b68-ae12-b32cb0c18089 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:30.510904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d00b16-a6aa-4b68-ae12-b32cb0c18089 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:30.511457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d00b16-a6aa-4b68-ae12-b32cb0c18089 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:30.513010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d00b16-a6aa-4b68-ae12-b32cb0c18089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:30.513975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d00b16-a6aa-4b68-ae12-b32cb0c18089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:30.515914 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d00b16-a6aa-4b68-ae12-b32cb0c18089 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:30.532273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 340/665] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:30.534028 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d00b16-a6aa-4b68-ae12-b32cb0c18089 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:30.575232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed6a2f-b5a6-4eff-af8a-61dc7d8dded5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:30.576741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbed6a2f-b5a6-4eff-af8a-61dc7d8dded5 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:30.656999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d00b16-a6aa-4b68-ae12-b32cb0c18089 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:30.673980 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:30.689941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 326/666] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:34:30.691497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed6a2f-b5a6-4eff-af8a-61dc7d8dded5 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:30.694504 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfd225-cbec-472c-8a20-86fd1952eb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:30.695626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfd225-cbec-472c-8a20-86fd1952eb3c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:30.695626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfd225-cbec-472c-8a20-86fd1952eb3c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:30.695626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfd225-cbec-472c-8a20-86fd1952eb3c None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:30.696260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfd225-cbec-472c-8a20-86fd1952eb3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:30.696561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfd225-cbec-472c-8a20-86fd1952eb3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:30.697959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dfd225-cbec-472c-8a20-86fd1952eb3c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:30.709838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 341/667] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:30.714769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfd225-cbec-472c-8a20-86fd1952eb3c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:30.733336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:30.799860 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbed6a2f-b5a6-4eff-af8a-61dc7d8dded5 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:30.824557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dfd225-cbec-472c-8a20-86fd1952eb3c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:30.837234 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a6fab-d708-454e-838f-29f1dc1a712d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:30.837886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a6fab-d708-454e-838f-29f1dc1a712d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:30.838168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a6fab-d708-454e-838f-29f1dc1a712d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:30.838436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7a6fab-d708-454e-838f-29f1dc1a712d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:30.846097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a6fab-d708-454e-838f-29f1dc1a712d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:30.847647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7a6fab-d708-454e-838f-29f1dc1a712d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-9d7a6fab-d708-454e-838f-29f1dc1a712d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:30.859216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:30.860130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:30.860995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:30.860995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:30.860995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:30.860995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:30.860995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:30.860995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:30.860995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:30.860995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:30.860995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 327/668] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:30.861470 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a6fab-d708-454e-838f-29f1dc1a712d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:30.877834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde93ae5-f8f4-4628-8af8-6ba591413adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:30.878494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde93ae5-f8f4-4628-8af8-6ba591413adb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:30.878809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde93ae5-f8f4-4628-8af8-6ba591413adb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:30.879084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde93ae5-f8f4-4628-8af8-6ba591413adb None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:30.892086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde93ae5-f8f4-4628-8af8-6ba591413adb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-fde93ae5-f8f4-4628-8af8-6ba591413adb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:30.896839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:30.897754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:30.898648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:30.898648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:30.898648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:30.898648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:30.898648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:30.898648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:30.898648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:30.898648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:30.898648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:30.898648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:30.898648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:30.898648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:30.898648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 342/669] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:30.900687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde93ae5-f8f4-4628-8af8-6ba591413adb None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:30.917746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:30.969916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7a6fab-d708-454e-838f-29f1dc1a712d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:30.982119 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:31.007112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde93ae5-f8f4-4628-8af8-6ba591413adb None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:31.007804 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab8e8-2691-4383-9f52-565feb97e52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:31.008438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab8e8-2691-4383-9f52-565feb97e52c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:31.008717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab8e8-2691-4383-9f52-565feb97e52c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:31.008993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab8e8-2691-4383-9f52-565feb97e52c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:31.019734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ab8e8-2691-4383-9f52-565feb97e52c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:31.021178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ab8e8-2691-4383-9f52-565feb97e52c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-406ab8e8-2691-4383-9f52-565feb97e52c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:31.035951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:31.037079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:31.038179 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:31.038179 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:31.038179 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:31.041553 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 328/670] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:31.042302 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab8e8-2691-4383-9f52-565feb97e52c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:31.048625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f88a37-e36d-4dea-a2de-d6652a30a83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:31.049321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f88a37-e36d-4dea-a2de-d6652a30a83a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:31.049713 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f88a37-e36d-4dea-a2de-d6652a30a83a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:31.050078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f88a37-e36d-4dea-a2de-d6652a30a83a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:31.057347 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f88a37-e36d-4dea-a2de-d6652a30a83a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:31.058904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f88a37-e36d-4dea-a2de-d6652a30a83a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-36f88a37-e36d-4dea-a2de-d6652a30a83a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:31.071141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:31.072196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:31.073167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:31.073167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:31.073167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:31.073167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:31.073167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:31.073167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:31.073167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:31.073167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:31.073167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 343/671] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:31.073167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f88a37-e36d-4dea-a2de-d6652a30a83a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:31.154606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:31.158225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab8e8-2691-4383-9f52-565feb97e52c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:31.179620 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f88a37-e36d-4dea-a2de-d6652a30a83a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:31.196607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaebe3-40e3-4042-b2de-636702830933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:31.197363 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaebe3-40e3-4042-b2de-636702830933 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:31.197832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaebe3-40e3-4042-b2de-636702830933 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:31.198295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaebe3-40e3-4042-b2de-636702830933 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:31.199447 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffaebe3-40e3-4042-b2de-636702830933 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:31.199969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffaebe3-40e3-4042-b2de-636702830933 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:31.202388 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffaebe3-40e3-4042-b2de-636702830933 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:31.221463 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 329/672] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:31.222821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaebe3-40e3-4042-b2de-636702830933 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:31.243296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbca33-e13f-457e-8759-f21cd0d25b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:31.244073 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbca33-e13f-457e-8759-f21cd0d25b71 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:31.244513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbca33-e13f-457e-8759-f21cd0d25b71 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:31.244957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bbca33-e13f-457e-8759-f21cd0d25b71 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:31.246053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bbca33-e13f-457e-8759-f21cd0d25b71 None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:31.246554 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bbca33-e13f-457e-8759-f21cd0d25b71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:31.249215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bbca33-e13f-457e-8759-f21cd0d25b71 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:31.254065 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:31.269323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 344/673] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:31.269323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbca33-e13f-457e-8759-f21cd0d25b71 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:31.345268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaebe3-40e3-4042-b2de-636702830933 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:31.372236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bbca33-e13f-457e-8759-f21cd0d25b71 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:31.378105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e671f2-9f9f-4d33-a029-e5212da331cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:31.378715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e671f2-9f9f-4d33-a029-e5212da331cb None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:31.378993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e671f2-9f9f-4d33-a029-e5212da331cb None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:31.379264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e671f2-9f9f-4d33-a029-e5212da331cb None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:31.379924 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e671f2-9f9f-4d33-a029-e5212da331cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:31.380224 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e671f2-9f9f-4d33-a029-e5212da331cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:31.381577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e671f2-9f9f-4d33-a029-e5212da331cb None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:31.392990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 330/674] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:31.393766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e671f2-9f9f-4d33-a029-e5212da331cb None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:31.405465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee4ae1-6637-4e4a-a1a2-dce4052c1e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:31.406090 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee4ae1-6637-4e4a-a1a2-dce4052c1e12 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:31.406366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee4ae1-6637-4e4a-a1a2-dce4052c1e12 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:31.406635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee4ae1-6637-4e4a-a1a2-dce4052c1e12 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:31.407307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee4ae1-6637-4e4a-a1a2-dce4052c1e12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:31.407608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee4ae1-6637-4e4a-a1a2-dce4052c1e12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:31.409021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee4ae1-6637-4e4a-a1a2-dce4052c1e12 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:31.416128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:31.421515 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 345/675] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:31.422490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee4ae1-6637-4e4a-a1a2-dce4052c1e12 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:31.481613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:31.495035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e671f2-9f9f-4d33-a029-e5212da331cb None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:31.524732 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee4ae1-6637-4e4a-a1a2-dce4052c1e12 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:31.529004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7caf8-1dfc-4b82-91cc-c0b5e546d29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:31.529601 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7caf8-1dfc-4b82-91cc-c0b5e546d29b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:31.529897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7caf8-1dfc-4b82-91cc-c0b5e546d29b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:31.530162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7caf8-1dfc-4b82-91cc-c0b5e546d29b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:31.530864 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7caf8-1dfc-4b82-91cc-c0b5e546d29b None admin] RBAC: Adding query filter params (role.id=928c1b8875334543ab431e4f7e2acee5, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=4e021a00524341359bc4b5a800a9bbf6) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:31.531163 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7caf8-1dfc-4b82-91cc-c0b5e546d29b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:31.532588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7caf8-1dfc-4b82-91cc-c0b5e546d29b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:31.557909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24baa9a-88d6-496a-af9b-3917d19a9fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:31.558522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24baa9a-88d6-496a-af9b-3917d19a9fae None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:31.558794 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24baa9a-88d6-496a-af9b-3917d19a9fae None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:31.559087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24baa9a-88d6-496a-af9b-3917d19a9fae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:31.568823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24baa9a-88d6-496a-af9b-3917d19a9fae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:31.570257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24baa9a-88d6-496a-af9b-3917d19a9fae None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:31.583054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 331/676] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:31 2022] GET /identity/v3/role_assignments?user.id=4e021a00524341359bc4b5a800a9bbf6&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=928c1b8875334543ab431e4f7e2acee5 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:31.584351 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d2973-3b62-49a5-a8c8-c48344114c90 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:31.584643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d2973-3b62-49a5-a8c8-c48344114c90 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-c24baa9a-88d6-496a-af9b-3917d19a9fae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:31.584906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:31.585831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:31.587122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:31.587122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:31.587122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:31.587309 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d2973-3b62-49a5-a8c8-c48344114c90 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:31.587381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 346/677] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:31.587381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24baa9a-88d6-496a-af9b-3917d19a9fae None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:31.588278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 332/678] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:31.589254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d2973-3b62-49a5-a8c8-c48344114c90 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:31.703667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24baa9a-88d6-496a-af9b-3917d19a9fae None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:31.705243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:31.717121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:31.723339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d2973-3b62-49a5-a8c8-c48344114c90 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:31.742464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a406cf-24b8-41aa-9531-a32ed89d8ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:31.743032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a406cf-24b8-41aa-9531-a32ed89d8ed1 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:31.743318 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a406cf-24b8-41aa-9531-a32ed89d8ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:31.743594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a406cf-24b8-41aa-9531-a32ed89d8ed1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:31.751488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a406cf-24b8-41aa-9531-a32ed89d8ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:31.752966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a406cf-24b8-41aa-9531-a32ed89d8ed1 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:31.759513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800b552-8853-46b2-b95f-c2e1f65eda25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:31.760126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800b552-8853-46b2-b95f-c2e1f65eda25 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:31.760405 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800b552-8853-46b2-b95f-c2e1f65eda25 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:31.760673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800b552-8853-46b2-b95f-c2e1f65eda25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:31.761457 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800b552-8853-46b2-b95f-c2e1f65eda25 None admin] RBAC: Adding query filter params (role.id=928c1b8875334543ab431e4f7e2acee5, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=7103ff9544524393a8c7175998dcae3b) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:31.761766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800b552-8853-46b2-b95f-c2e1f65eda25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:31.763294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8800b552-8853-46b2-b95f-c2e1f65eda25 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-c9a406cf-24b8-41aa-9531-a32ed89d8ed1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:31.767010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:31.768051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:31.769123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:31.769123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:31.769123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:31.769123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:31.769123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:31.769123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:31.769123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:31.769123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:31.771463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 347/679] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:31.772472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a406cf-24b8-41aa-9531-a32ed89d8ed1 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:31.774969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 333/680] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:31 2022] GET /identity/v3/role_assignments?user.id=7103ff9544524393a8c7175998dcae3b&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=928c1b8875334543ab431e4f7e2acee5 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:31.776373 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d64c0-eedc-4b85-9e3c-d8949a17e59e None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:31.776785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d64c0-eedc-4b85-9e3c-d8949a17e59e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:31.777205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d64c0-eedc-4b85-9e3c-d8949a17e59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:31.801953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-ce9d64c0-eedc-4b85-9e3c-d8949a17e59e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:31.804380 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ce9d64c0-eedc-4b85-9e3c-d8949a17e59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:31.905682 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 334/681] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:31.907253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d64c0-eedc-4b85-9e3c-d8949a17e59e None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:31.920632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a406cf-24b8-41aa-9531-a32ed89d8ed1 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:31.993281 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f15e99-08b6-4862-a551-013bafedb059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:31.994741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15e99-08b6-4862-a551-013bafedb059 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:31.995177 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15e99-08b6-4862-a551-013bafedb059 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:31.995608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15e99-08b6-4862-a551-013bafedb059 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:31.997010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f15e99-08b6-4862-a551-013bafedb059 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:31.997555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f15e99-08b6-4862-a551-013bafedb059 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:31.999189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f15e99-08b6-4862-a551-013bafedb059 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:32.015638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 348/682] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:32.020963 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f15e99-08b6-4862-a551-013bafedb059 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:32.031422 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:32.057922 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:32.065002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d64c0-eedc-4b85-9e3c-d8949a17e59e None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:32.111991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167e33c-a0a0-4453-838f-18e0efb95635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:32.112965 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167e33c-a0a0-4453-838f-18e0efb95635 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:32.114024 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167e33c-a0a0-4453-838f-18e0efb95635 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:32.114515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2167e33c-a0a0-4453-838f-18e0efb95635 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:32.115826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167e33c-a0a0-4453-838f-18e0efb95635 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:32.116609 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167e33c-a0a0-4453-838f-18e0efb95635 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:32.118569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2167e33c-a0a0-4453-838f-18e0efb95635 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:32.135160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 335/683] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:32.141509 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167e33c-a0a0-4453-838f-18e0efb95635 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:32.141884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f15e99-08b6-4862-a551-013bafedb059 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:32.211113 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f736a-8aa1-464a-89bd-68012a0c3ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:32.211746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f736a-8aa1-464a-89bd-68012a0c3ca7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:32.212026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f736a-8aa1-464a-89bd-68012a0c3ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:32.212299 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81f736a-8aa1-464a-89bd-68012a0c3ca7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:32.221963 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f736a-8aa1-464a-89bd-68012a0c3ca7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:32.222294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81f736a-8aa1-464a-89bd-68012a0c3ca7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-e81f736a-8aa1-464a-89bd-68012a0c3ca7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:32.235648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:32.236839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:32.238127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:32.238127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:32.238127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:32.238127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:32.238127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:32.238127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:32.238127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:32.238127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:32.238127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 19:34:32.238127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:32.238127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 349/684] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:34:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:32.243465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f736a-8aa1-464a-89bd-68012a0c3ca7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:32.275340 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2167e33c-a0a0-4453-838f-18e0efb95635 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:32.313642 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae606b43-64cc-4981-872a-03d8729ae866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:32.314730 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae606b43-64cc-4981-872a-03d8729ae866 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:32.315209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae606b43-64cc-4981-872a-03d8729ae866 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:32.315691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae606b43-64cc-4981-872a-03d8729ae866 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:32.316698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae606b43-64cc-4981-872a-03d8729ae866 None admin] RBAC: Adding query filter params (role.id=928c1b8875334543ab431e4f7e2acee5, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=8a2c9eec08b142138569043e3b34c6a8) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:32.317515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae606b43-64cc-4981-872a-03d8729ae866 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:32.319221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae606b43-64cc-4981-872a-03d8729ae866 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:32.334015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 336/685] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:32 2022] GET /identity/v3/role_assignments?user.id=8a2c9eec08b142138569043e3b34c6a8&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=928c1b8875334543ab431e4f7e2acee5 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:32.335333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae606b43-64cc-4981-872a-03d8729ae866 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:32.338696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:32.354389 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:32.382415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81f736a-8aa1-464a-89bd-68012a0c3ca7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:32.454154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28d4ce-d9f5-47c5-aff3-73e25367336a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:32.454943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28d4ce-d9f5-47c5-aff3-73e25367336a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:32.455214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28d4ce-d9f5-47c5-aff3-73e25367336a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:32.455486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec28d4ce-d9f5-47c5-aff3-73e25367336a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:32.486864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec28d4ce-d9f5-47c5-aff3-73e25367336a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-ec28d4ce-d9f5-47c5-aff3-73e25367336a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:32.492355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:32.493332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:32.494265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:32.494265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:32.494265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:32.494265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:32.494265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:32.494265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:32.494265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:32.494265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:32.494265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:32.494265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:32.494265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:34:32.494265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:32.496117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 350/686] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:34:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:32.502546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28d4ce-d9f5-47c5-aff3-73e25367336a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:32.504687 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae606b43-64cc-4981-872a-03d8729ae866 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:32.563559 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb85368-8428-445f-b6d6-5729a1fe4e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:32.565941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb85368-8428-445f-b6d6-5729a1fe4e21 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:32.566491 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb85368-8428-445f-b6d6-5729a1fe4e21 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:32.568129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb85368-8428-445f-b6d6-5729a1fe4e21 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:32.569092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb85368-8428-445f-b6d6-5729a1fe4e21 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:32.571006 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb85368-8428-445f-b6d6-5729a1fe4e21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:32.574478 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb85368-8428-445f-b6d6-5729a1fe4e21 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:32.597775 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 337/687] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Sep 14 19:34:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:32.604075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb85368-8428-445f-b6d6-5729a1fe4e21 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:32.683724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec28d4ce-d9f5-47c5-aff3-73e25367336a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:32.690956 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:32.736721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a92a3-3ad1-4301-bcf4-169393eda2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:32.737704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a92a3-3ad1-4301-bcf4-169393eda2f7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:32.738226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a92a3-3ad1-4301-bcf4-169393eda2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:32.745884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7a92a3-3ad1-4301-bcf4-169393eda2f7 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:32.747642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a92a3-3ad1-4301-bcf4-169393eda2f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:32.748161 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a92a3-3ad1-4301-bcf4-169393eda2f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:32.749301 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb85368-8428-445f-b6d6-5729a1fe4e21 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:32.750365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7a92a3-3ad1-4301-bcf4-169393eda2f7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:32.767756 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 351/688] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:32.773286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a92a3-3ad1-4301-bcf4-169393eda2f7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:32.797015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abd97a-d839-45a7-aeb2-f474557aa8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:32.797638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abd97a-d839-45a7-aeb2-f474557aa8b3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:32.797942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abd97a-d839-45a7-aeb2-f474557aa8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:32.798215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abd97a-d839-45a7-aeb2-f474557aa8b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:32.802268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:32.883094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abd97a-d839-45a7-aeb2-f474557aa8b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:32.884741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abd97a-d839-45a7-aeb2-f474557aa8b3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:32.958729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:32.993486 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 338/689] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:34:33.002015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7a92a3-3ad1-4301-bcf4-169393eda2f7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:33.002466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abd97a-d839-45a7-aeb2-f474557aa8b3 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:33.064853 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fe2f8-f74d-4958-b1f6-096345647361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:33.065494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fe2f8-f74d-4958-b1f6-096345647361 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:33.065770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fe2f8-f74d-4958-b1f6-096345647361 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:33.066088 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fe2f8-f74d-4958-b1f6-096345647361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:33.088906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9fe2f8-f74d-4958-b1f6-096345647361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:33.090426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9fe2f8-f74d-4958-b1f6-096345647361 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-db9fe2f8-f74d-4958-b1f6-096345647361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:33.108966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:33.109993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:33.110891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:33.110891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:33.110891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:33.110891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 352/690] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:33.116096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fe2f8-f74d-4958-b1f6-096345647361 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:33.187180 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:33.211746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abd97a-d839-45a7-aeb2-f474557aa8b3 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:33.232960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fe2f8-f74d-4958-b1f6-096345647361 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:33.269507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d78432-3011-40ba-af46-8809d6182ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:33.270483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d78432-3011-40ba-af46-8809d6182ded None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:33.270993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d78432-3011-40ba-af46-8809d6182ded None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:33.271465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d78432-3011-40ba-af46-8809d6182ded None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:33.272313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d78432-3011-40ba-af46-8809d6182ded None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:33.272816 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d78432-3011-40ba-af46-8809d6182ded None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:33.274480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d78432-3011-40ba-af46-8809d6182ded None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:33.282203 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253f0ee-9a7b-4740-a60a-066bee2466e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:33.282777 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253f0ee-9a7b-4740-a60a-066bee2466e6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:33.283054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253f0ee-9a7b-4740-a60a-066bee2466e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:33.283324 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253f0ee-9a7b-4740-a60a-066bee2466e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:33.288784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 353/691] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:33.294481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d78432-3011-40ba-af46-8809d6182ded None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:33.304540 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253f0ee-9a7b-4740-a60a-066bee2466e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:33.307934 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253f0ee-9a7b-4740-a60a-066bee2466e6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-0253f0ee-9a7b-4740-a60a-066bee2466e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:33.336401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:33.337489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:33.338424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:33.338424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:33.338424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:33.347239 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 339/692] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:33.350279 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253f0ee-9a7b-4740-a60a-066bee2466e6 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:33.379757 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:33.511392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253f0ee-9a7b-4740-a60a-066bee2466e6 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:33.519983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d78432-3011-40ba-af46-8809d6182ded None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:33.551294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e57de-0d6b-4344-9a8d-892575ef6176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:33.552278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e57de-0d6b-4344-9a8d-892575ef6176 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:33.552780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e57de-0d6b-4344-9a8d-892575ef6176 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:33.553257 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6e57de-0d6b-4344-9a8d-892575ef6176 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:33.554346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e57de-0d6b-4344-9a8d-892575ef6176 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:33.554853 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e57de-0d6b-4344-9a8d-892575ef6176 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:33.556870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6e57de-0d6b-4344-9a8d-892575ef6176 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:33.571709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 340/693] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:33.581303 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e57de-0d6b-4344-9a8d-892575ef6176 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:33.582974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:33.610158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f908f-d9c9-43d8-815a-6820f1ac4009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:33.611176 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f908f-d9c9-43d8-815a-6820f1ac4009 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:33.613203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f908f-d9c9-43d8-815a-6820f1ac4009 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:33.613715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f908f-d9c9-43d8-815a-6820f1ac4009 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:33.630141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f908f-d9c9-43d8-815a-6820f1ac4009 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:33.632776 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f908f-d9c9-43d8-815a-6820f1ac4009 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-a22f908f-d9c9-43d8-815a-6820f1ac4009 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:33.659245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:33.660275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:33.661251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:33.661251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:33.661251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:33.661251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:33.661251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:33.661251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:33.661251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:33.661251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:33.661251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:34:33.661251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:33.663599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 354/694] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 19:34:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 370 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:33.671924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f908f-d9c9-43d8-815a-6820f1ac4009 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:33.750152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:33.796454 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f908f-d9c9-43d8-815a-6820f1ac4009 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:33.817775 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6e57de-0d6b-4344-9a8d-892575ef6176 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:33.833257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb2d67-6f84-40d5-ab9f-e6a3d6a1c4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:33.834360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb2d67-6f84-40d5-ab9f-e6a3d6a1c4e3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:33.834741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb2d67-6f84-40d5-ab9f-e6a3d6a1c4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:33.835119 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb2d67-6f84-40d5-ab9f-e6a3d6a1c4e3 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:33.835944 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb2d67-6f84-40d5-ab9f-e6a3d6a1c4e3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:33.836353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb2d67-6f84-40d5-ab9f-e6a3d6a1c4e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:33.845820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb2d67-6f84-40d5-ab9f-e6a3d6a1c4e3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:33.870075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 355/695] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 19:34:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:33.874153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb2d67-6f84-40d5-ab9f-e6a3d6a1c4e3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:33.886250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75668d-8af1-41f8-84b1-1dcb53ef8ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:33.887017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75668d-8af1-41f8-84b1-1dcb53ef8ef4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:33.887778 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75668d-8af1-41f8-84b1-1dcb53ef8ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:33.888152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc75668d-8af1-41f8-84b1-1dcb53ef8ef4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:33.901585 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75668d-8af1-41f8-84b1-1dcb53ef8ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:33.903373 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc75668d-8af1-41f8-84b1-1dcb53ef8ef4 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-dc75668d-8af1-41f8-84b1-1dcb53ef8ef4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:33.916755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:33.917895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:33.919506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:33.919506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:33.919506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:33.919506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:33.919506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:33.919506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:33.919506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:33.919506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:33.921259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 341/696] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:33.927996 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75668d-8af1-41f8-84b1-1dcb53ef8ef4 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:33.959173 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:34.081883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb2d67-6f84-40d5-ab9f-e6a3d6a1c4e3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:34.084426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc75668d-8af1-41f8-84b1-1dcb53ef8ef4 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:34.136940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:34.141784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b1247-ae9c-47ec-8a84-e068f37fcce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:34.142513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b1247-ae9c-47ec-8a84-e068f37fcce4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:34.146298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b1247-ae9c-47ec-8a84-e068f37fcce4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:34.146298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b1247-ae9c-47ec-8a84-e068f37fcce4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:34.146496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ff2f9-acaa-4967-ad82-d762b79f16e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:34.146496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ff2f9-acaa-4967-ad82-d762b79f16e6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:34.146496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ff2f9-acaa-4967-ad82-d762b79f16e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:34.146496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ff2f9-acaa-4967-ad82-d762b79f16e6 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:34.146788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ff2f9-acaa-4967-ad82-d762b79f16e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:34.146788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ff2f9-acaa-4967-ad82-d762b79f16e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:34.148712 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ff2f9-acaa-4967-ad82-d762b79f16e6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:34.170185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 342/697] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:34.172345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541826cc-4eda-4344-8b4c-6f86beed4cd7 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:34.172908 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541826cc-4eda-4344-8b4c-6f86beed4cd7 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:34.174356 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541826cc-4eda-4344-8b4c-6f86beed4cd7 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:34.176695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 343/698] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:34.177776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541826cc-4eda-4344-8b4c-6f86beed4cd7 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:34.232417 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b1247-ae9c-47ec-8a84-e068f37fcce4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:34.234082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b1247-ae9c-47ec-8a84-e068f37fcce4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:34.286306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:34.326802 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541826cc-4eda-4344-8b4c-6f86beed4cd7 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:34.356896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 356/699] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 19:34:34.359327 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb3bc3-4212-4608-bbc4-2e14b7ae8dba None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:34.359857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb3bc3-4212-4608-bbc4-2e14b7ae8dba None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:34.360287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb3bc3-4212-4608-bbc4-2e14b7ae8dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:34.372380 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d8f93-a960-470c-ac97-faea95d91a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:34.373138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d8f93-a960-470c-ac97-faea95d91a07 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:34.373515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d8f93-a960-470c-ac97-faea95d91a07 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:34.373910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223d8f93-a960-470c-ac97-faea95d91a07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:34.375033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d8f93-a960-470c-ac97-faea95d91a07 None admin] RBAC: Adding query filter params (role.id=928c1b8875334543ab431e4f7e2acee5, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=e3d2f1ab1bfd435183e33168400c0b17) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:34.375729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d8f93-a960-470c-ac97-faea95d91a07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:34.377015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223d8f93-a960-470c-ac97-faea95d91a07 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:34.382813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-cfdb3bc3-4212-4608-bbc4-2e14b7ae8dba None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:34.383925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 344/700] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:34 2022] GET /identity/v3/role_assignments?user.id=e3d2f1ab1bfd435183e33168400c0b17&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=928c1b8875334543ab431e4f7e2acee5 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:34.389821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cfdb3bc3-4212-4608-bbc4-2e14b7ae8dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:34.389994 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d8f93-a960-470c-ac97-faea95d91a07 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:34.453131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:34.475785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 357/701] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:34.477081 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556735b-f62e-404d-9c1b-039e99d8514b None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:34.477430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556735b-f62e-404d-9c1b-039e99d8514b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:34.477701 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8556735b-f62e-404d-9c1b-039e99d8514b None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:34.479493 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 358/702] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:34.485769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65766ed2-c964-43be-9b5d-99c7474ca082 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:34.486146 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65766ed2-c964-43be-9b5d-99c7474ca082 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:34.486416 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65766ed2-c964-43be-9b5d-99c7474ca082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:34.510856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-65766ed2-c964-43be-9b5d-99c7474ca082 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:34.512305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-65766ed2-c964-43be-9b5d-99c7474ca082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:34.541330 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223d8f93-a960-470c-ac97-faea95d91a07 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:34.624155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab3ea8-75ce-47b5-a1e5-e61542732c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:34.624707 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab3ea8-75ce-47b5-a1e5-e61542732c64 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:34.624987 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab3ea8-75ce-47b5-a1e5-e61542732c64 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:34.625260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab3ea8-75ce-47b5-a1e5-e61542732c64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:34.638186 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 359/703] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:34.641755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab3ea8-75ce-47b5-a1e5-e61542732c64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:34.643237 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab3ea8-75ce-47b5-a1e5-e61542732c64 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-abab3ea8-75ce-47b5-a1e5-e61542732c64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:34.659355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:34.660428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:34.661979 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:34.661979 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:34.661979 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:34.661979 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 345/704] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:34.666258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65766ed2-c964-43be-9b5d-99c7474ca082 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:34.677859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:34.776241 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65766ed2-c964-43be-9b5d-99c7474ca082 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:34.784676 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab3ea8-75ce-47b5-a1e5-e61542732c64 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:34.818657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b27cd5-219b-4834-bccb-6e2239cfe2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:34.819275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b27cd5-219b-4834-bccb-6e2239cfe2ca None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:34.819582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b27cd5-219b-4834-bccb-6e2239cfe2ca None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:34.819852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b27cd5-219b-4834-bccb-6e2239cfe2ca None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:34.820942 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b27cd5-219b-4834-bccb-6e2239cfe2ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:34.821246 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b27cd5-219b-4834-bccb-6e2239cfe2ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:34.822932 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b27cd5-219b-4834-bccb-6e2239cfe2ca None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:34.845233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 360/705] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:34.855063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b27cd5-219b-4834-bccb-6e2239cfe2ca None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:34.927156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab3ea8-75ce-47b5-a1e5-e61542732c64 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:34.939419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:34.947746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:34.971652 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ccabf-525a-4862-9fe6-e2e5c0265aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:34.972917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ccabf-525a-4862-9fe6-e2e5c0265aa7 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:34.973201 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ccabf-525a-4862-9fe6-e2e5c0265aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:34.973483 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ccabf-525a-4862-9fe6-e2e5c0265aa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:34.987497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ccabf-525a-4862-9fe6-e2e5c0265aa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:34.989616 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ccabf-525a-4862-9fe6-e2e5c0265aa7 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:34.994935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b27cd5-219b-4834-bccb-6e2239cfe2ca None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-b14ccabf-525a-4862-9fe6-e2e5c0265aa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:35.007846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:35.008911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:35.010188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:35.010188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:35.010188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:35.010188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 346/706] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:35.014157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ccabf-525a-4862-9fe6-e2e5c0265aa7 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:35.039065 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe40011-82fc-49bb-8f52-a7466b91f600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:35.039935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe40011-82fc-49bb-8f52-a7466b91f600 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:35.040218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe40011-82fc-49bb-8f52-a7466b91f600 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:35.040491 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe40011-82fc-49bb-8f52-a7466b91f600 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:35.048052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe40011-82fc-49bb-8f52-a7466b91f600 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:35.049481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe40011-82fc-49bb-8f52-a7466b91f600 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-8fe40011-82fc-49bb-8f52-a7466b91f600 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:35.061332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:35.062420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:35.063498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:35.063498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:35.063498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:35.063498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:35.063498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:35.063498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:35.063498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:35.063498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:35.063498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 361/707] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:35.064079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe40011-82fc-49bb-8f52-a7466b91f600 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:35.128513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ccabf-525a-4862-9fe6-e2e5c0265aa7 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:35.166863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7645eae-810b-4dc6-8496-6b43b280d973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:35.167501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7645eae-810b-4dc6-8496-6b43b280d973 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:35.167784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7645eae-810b-4dc6-8496-6b43b280d973 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:35.168057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7645eae-810b-4dc6-8496-6b43b280d973 None admin] PATH_INFO: `/v3/services/network` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:35.169151 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7645eae-810b-4dc6-8496-6b43b280d973 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:35.170539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7645eae-810b-4dc6-8496-6b43b280d973 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:35.171423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe40011-82fc-49bb-8f52-a7466b91f600 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-c7645eae-810b-4dc6-8496-6b43b280d973 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:35.181477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:35.182708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:35.183548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 347/708] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:35 2022] GET /identity/v3/services/network => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:35.184426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0cf34-e093-4821-9a7b-ad23288c7343 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:35.184426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0cf34-e093-4821-9a7b-ad23288c7343 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:35.184426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0cf34-e093-4821-9a7b-ad23288c7343 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:35.185949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 348/709] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:35.190272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0cf34-e093-4821-9a7b-ad23288c7343 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:35.201525 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:35.214496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a19bf-e3f7-4d62-b871-e70a96654212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:35.215456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a19bf-e3f7-4d62-b871-e70a96654212 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:35.215701 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a19bf-e3f7-4d62-b871-e70a96654212 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:35.216052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a19bf-e3f7-4d62-b871-e70a96654212 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:35.216763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a19bf-e3f7-4d62-b871-e70a96654212 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:35.217029 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:35.217195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a19bf-e3f7-4d62-b871-e70a96654212 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:35.218830 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a19bf-e3f7-4d62-b871-e70a96654212 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:35.235748 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 362/710] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:35.235748 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a19bf-e3f7-4d62-b871-e70a96654212 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:35.308357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0cf34-e093-4821-9a7b-ad23288c7343 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:35.343469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a19bf-e3f7-4d62-b871-e70a96654212 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:35.349310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfee5a-8d9b-48d5-8f51-6ecf9d9951a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:35.349950 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfee5a-8d9b-48d5-8f51-6ecf9d9951a5 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:35.350236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfee5a-8d9b-48d5-8f51-6ecf9d9951a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:35.350499 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfee5a-8d9b-48d5-8f51-6ecf9d9951a5 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:35.351180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfee5a-8d9b-48d5-8f51-6ecf9d9951a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:35.351486 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfee5a-8d9b-48d5-8f51-6ecf9d9951a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:35.352925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdfee5a-8d9b-48d5-8f51-6ecf9d9951a5 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:35.370737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 349/711] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:35.370737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11047f-05a6-41c2-a678-b746e7df6c80 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:35.370737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11047f-05a6-41c2-a678-b746e7df6c80 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:35.370737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11047f-05a6-41c2-a678-b746e7df6c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:35.379761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff8364-2a48-4be8-a06c-7c2a588b931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:35.380377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff8364-2a48-4be8-a06c-7c2a588b931a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:35.380738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff8364-2a48-4be8-a06c-7c2a588b931a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:35.381075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff8364-2a48-4be8-a06c-7c2a588b931a None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:35.381749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff8364-2a48-4be8-a06c-7c2a588b931a None admin] RBAC: Adding query filter params (name=network) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:35.382135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff8364-2a48-4be8-a06c-7c2a588b931a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:35.383544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bff8364-2a48-4be8-a06c-7c2a588b931a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:35.388076 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-2e11047f-05a6-41c2-a678-b746e7df6c80 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:35.388190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 363/712] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 19:34:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:35.389236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2e11047f-05a6-41c2-a678-b746e7df6c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:35.393084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff8364-2a48-4be8-a06c-7c2a588b931a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:35.440245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:35.459583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 350/713] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:35.460650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11047f-05a6-41c2-a678-b746e7df6c80 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:35.482653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:35.511343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bff8364-2a48-4be8-a06c-7c2a588b931a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:35.556370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f43acc-ac08-4122-81b8-fd4e13e1e8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:35.557408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f43acc-ac08-4122-81b8-fd4e13e1e8b8 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:35.557952 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f43acc-ac08-4122-81b8-fd4e13e1e8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:35.558409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f43acc-ac08-4122-81b8-fd4e13e1e8b8 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:35.559755 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f43acc-ac08-4122-81b8-fd4e13e1e8b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:35.561478 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f43acc-ac08-4122-81b8-fd4e13e1e8b8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:35.591941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11047f-05a6-41c2-a678-b746e7df6c80 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:35.628463 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af19f71-67b2-435b-9e28-1d35aefbdb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:35.629375 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af19f71-67b2-435b-9e28-1d35aefbdb1e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:35.629916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af19f71-67b2-435b-9e28-1d35aefbdb1e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:35.630401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af19f71-67b2-435b-9e28-1d35aefbdb1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:35.631459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af19f71-67b2-435b-9e28-1d35aefbdb1e None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=2a4284b59f804dc59674d621d87634bc) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:35.632009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af19f71-67b2-435b-9e28-1d35aefbdb1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:35.633604 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af19f71-67b2-435b-9e28-1d35aefbdb1e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:35.646243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 351/714] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:35 2022] GET /identity/v3/role_assignments?user.id=2a4284b59f804dc59674d621d87634bc&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:35.647487 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af19f71-67b2-435b-9e28-1d35aefbdb1e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:35.649170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 364/715] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:34:35 2022] POST /identity/v3/users => generated 254 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:34:35.749551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f43acc-ac08-4122-81b8-fd4e13e1e8b8 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:35.798223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:35.862389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af19f71-67b2-435b-9e28-1d35aefbdb1e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:35.908097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0747fbf-ab0d-4f39-8b74-58e712fc946c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:35.908820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0747fbf-ab0d-4f39-8b74-58e712fc946c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:35.909492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0747fbf-ab0d-4f39-8b74-58e712fc946c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:35.909886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0747fbf-ab0d-4f39-8b74-58e712fc946c None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:35.910588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0747fbf-ab0d-4f39-8b74-58e712fc946c None admin] RBAC: Adding query filter params (type=network) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:35.910963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0747fbf-ab0d-4f39-8b74-58e712fc946c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:35.912679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0747fbf-ab0d-4f39-8b74-58e712fc946c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:35.920399 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f43acc-ac08-4122-81b8-fd4e13e1e8b8 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:35.920952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 352/716] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 19:34:35 2022] GET /identity/v3/services?type=network => generated 124 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:35.926270 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:35.969484 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5c89a-8ea6-4dc5-b89e-c39d7fb549ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:35.970447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5c89a-8ea6-4dc5-b89e-c39d7fb549ef None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:35.970830 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5c89a-8ea6-4dc5-b89e-c39d7fb549ef None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:35.971206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5c89a-8ea6-4dc5-b89e-c39d7fb549ef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:35.971815 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5c89a-8ea6-4dc5-b89e-c39d7fb549ef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:35.973298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5c89a-8ea6-4dc5-b89e-c39d7fb549ef None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-3ae5c89a-8ea6-4dc5-b89e-c39d7fb549ef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:35.986684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:35.987746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:35.988896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 365/717] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Sep 14 19:34:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:35.997261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0747fbf-ab0d-4f39-8b74-58e712fc946c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:36.116338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0747fbf-ab0d-4f39-8b74-58e712fc946c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:36.156393 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f995721-0e62-4ce9-bab2-6092c35bc8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:36.157009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f995721-0e62-4ce9-bab2-6092c35bc8f7 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:36.157342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f995721-0e62-4ce9-bab2-6092c35bc8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:36.157620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f995721-0e62-4ce9-bab2-6092c35bc8f7 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:36.158334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f995721-0e62-4ce9-bab2-6092c35bc8f7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:36.158586 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f995721-0e62-4ce9-bab2-6092c35bc8f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:36.160046 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f995721-0e62-4ce9-bab2-6092c35bc8f7 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:36.165039 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 353/718] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Sep 14 19:34:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:36.170969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5c89a-8ea6-4dc5-b89e-c39d7fb549ef None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:36.280633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5c89a-8ea6-4dc5-b89e-c39d7fb549ef None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:36.315476 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4deca-75e9-4a73-af16-2a7bebdb7a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:36.316083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4deca-75e9-4a73-af16-2a7bebdb7a8c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:36.316364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4deca-75e9-4a73-af16-2a7bebdb7a8c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:36.316640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4deca-75e9-4a73-af16-2a7bebdb7a8c None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:36.317519 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4deca-75e9-4a73-af16-2a7bebdb7a8c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:36.317855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4deca-75e9-4a73-af16-2a7bebdb7a8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:36.319402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4deca-75e9-4a73-af16-2a7bebdb7a8c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:36.325429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 366/719] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Sep 14 19:34:36 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:37.127576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58512d3-029a-4a18-9361-a0e04284e51b None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:37.128088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58512d3-029a-4a18-9361-a0e04284e51b None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:37.128088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58512d3-029a-4a18-9361-a0e04284e51b None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:37.129380 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 354/720] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:37.144299 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8ff80-fb14-42e6-ab5c-b337d93372ad None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:37.144585 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8ff80-fb14-42e6-ab5c-b337d93372ad None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:37.144863 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8ff80-fb14-42e6-ab5c-b337d93372ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:37.174787 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7ef8ff80-fb14-42e6-ab5c-b337d93372ad None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:37.175992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7ef8ff80-fb14-42e6-ab5c-b337d93372ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:37.204690 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:37.262471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 367/721] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:37.650558 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58512d3-029a-4a18-9361-a0e04284e51b None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:37.660124 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:37.765403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58512d3-029a-4a18-9361-a0e04284e51b None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:37.802267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9dcdf-f1c5-4acd-8903-f4f9e8faffb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:37.802921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9dcdf-f1c5-4acd-8903-f4f9e8faffb3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:37.803197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9dcdf-f1c5-4acd-8903-f4f9e8faffb3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:37.803469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a9dcdf-f1c5-4acd-8903-f4f9e8faffb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:37.818344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a9dcdf-f1c5-4acd-8903-f4f9e8faffb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-65a9dcdf-f1c5-4acd-8903-f4f9e8faffb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:37.823830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:37.825374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:37.826258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:37.826258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:37.826258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:37.826258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:37.826258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:37.826258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:37.826258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:37.826258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:37.826258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:37.826258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:37.826258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:34:37.826258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:37.828600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 355/722] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:34:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:37.831746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8ff80-fb14-42e6-ab5c-b337d93372ad None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:37.959502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8ff80-fb14-42e6-ab5c-b337d93372ad None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:38.001842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f659d-7130-4c9d-9e53-bfb84f01aca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:38.002735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f659d-7130-4c9d-9e53-bfb84f01aca3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:38.003162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f659d-7130-4c9d-9e53-bfb84f01aca3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:38.003582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8f659d-7130-4c9d-9e53-bfb84f01aca3 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:38.004382 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f659d-7130-4c9d-9e53-bfb84f01aca3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:38.004857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f659d-7130-4c9d-9e53-bfb84f01aca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:38.006410 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8f659d-7130-4c9d-9e53-bfb84f01aca3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:38.020152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 368/723] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:38.021766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9dcdf-f1c5-4acd-8903-f4f9e8faffb3 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:38.105079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:38.146280 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a9dcdf-f1c5-4acd-8903-f4f9e8faffb3 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:38.192073 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8ba4b-dbfd-4f53-a69d-5141cd429faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:38.193045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8ba4b-dbfd-4f53-a69d-5141cd429faf None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:38.193329 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8ba4b-dbfd-4f53-a69d-5141cd429faf None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:38.193600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8ba4b-dbfd-4f53-a69d-5141cd429faf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:38.211819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8ba4b-dbfd-4f53-a69d-5141cd429faf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:38.213287 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8ba4b-dbfd-4f53-a69d-5141cd429faf None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-cca8ba4b-dbfd-4f53-a69d-5141cd429faf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:38.232897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:38.233977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:38.234867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:38.234867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:38.234867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:38.234867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 356/724] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:38.239865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f659d-7130-4c9d-9e53-bfb84f01aca3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:38.267813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:38.370169 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8f659d-7130-4c9d-9e53-bfb84f01aca3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:38.407037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93469014-bc4c-44ba-ac16-81d110da1590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:38.407643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93469014-bc4c-44ba-ac16-81d110da1590 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:38.407925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93469014-bc4c-44ba-ac16-81d110da1590 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:38.408194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93469014-bc4c-44ba-ac16-81d110da1590 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:38.408825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93469014-bc4c-44ba-ac16-81d110da1590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:38.409125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93469014-bc4c-44ba-ac16-81d110da1590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:38.410545 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93469014-bc4c-44ba-ac16-81d110da1590 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:38.424101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8c732-e422-43e0-b492-6f442539b7e8 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:38.424383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8c732-e422-43e0-b492-6f442539b7e8 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:38.424658 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8c732-e422-43e0-b492-6f442539b7e8 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:38.426079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 357/725] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:34:38.429669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 369/726] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:38.430982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aee796-ad0c-4849-9917-5afec3c97502 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:38.431267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aee796-ad0c-4849-9917-5afec3c97502 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:38.431537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aee796-ad0c-4849-9917-5afec3c97502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:38.437619 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93469014-bc4c-44ba-ac16-81d110da1590 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:38.461155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-90aee796-ad0c-4849-9917-5afec3c97502 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:38.463761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-90aee796-ad0c-4849-9917-5afec3c97502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:38.592068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:38.617093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 358/727] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:38.623745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93469014-bc4c-44ba-ac16-81d110da1590 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:38.659582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354eb7e8-a9c1-4bff-a92c-1a5aa3205e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:38.660260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354eb7e8-a9c1-4bff-a92c-1a5aa3205e90 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:38.660544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354eb7e8-a9c1-4bff-a92c-1a5aa3205e90 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:38.660817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354eb7e8-a9c1-4bff-a92c-1a5aa3205e90 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:38.668471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354eb7e8-a9c1-4bff-a92c-1a5aa3205e90 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:38.669999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354eb7e8-a9c1-4bff-a92c-1a5aa3205e90 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-354eb7e8-a9c1-4bff-a92c-1a5aa3205e90 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:38.682175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:38.683164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:38.684725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:38.684725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:38.684725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:38.684725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:38.684725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:38.684725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:38.684725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:38.684725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:38.684725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 19:34:38.684725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:38.688100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 370/728] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:34:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:38.690197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aee796-ad0c-4849-9917-5afec3c97502 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:38.691597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303a5b6-1143-4dc7-a58c-f69e8d99c6bb None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:38.691880 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303a5b6-1143-4dc7-a58c-f69e8d99c6bb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:38.692163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303a5b6-1143-4dc7-a58c-f69e8d99c6bb None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:38.694047 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 371/729] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:38.699046 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236340e8-c3db-4be9-92c3-1000dd1a1d4a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:38.699332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236340e8-c3db-4be9-92c3-1000dd1a1d4a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:38.700162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236340e8-c3db-4be9-92c3-1000dd1a1d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:38.734610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-236340e8-c3db-4be9-92c3-1000dd1a1d4a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:38.735745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-236340e8-c3db-4be9-92c3-1000dd1a1d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:38.760736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:38.824699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aee796-ad0c-4849-9917-5afec3c97502 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:38.829417 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 372/730] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:34:38.830987 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4bd62-3ae6-47db-9447-d3e36d3ecd09 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:38.831542 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4bd62-3ae6-47db-9447-d3e36d3ecd09 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:38.832020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4bd62-3ae6-47db-9447-d3e36d3ecd09 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:38.838734 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 373/731] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:38.840557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc466de-150b-4694-bec2-c2120179b1d0 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:38.841049 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc466de-150b-4694-bec2-c2120179b1d0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:38.842038 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc466de-150b-4694-bec2-c2120179b1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:38.872884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6dc466de-150b-4694-bec2-c2120179b1d0 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:38.874782 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6dc466de-150b-4694-bec2-c2120179b1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:38.896311 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1480744-d147-4790-a701-5f55c0e1969a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:38.897396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1480744-d147-4790-a701-5f55c0e1969a None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:38.897949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1480744-d147-4790-a701-5f55c0e1969a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:38.898245 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1480744-d147-4790-a701-5f55c0e1969a None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:38.898985 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1480744-d147-4790-a701-5f55c0e1969a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:38.899592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1480744-d147-4790-a701-5f55c0e1969a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:38.901406 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1480744-d147-4790-a701-5f55c0e1969a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:38.941047 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 359/732] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Sep 14 19:34:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:38.945638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1480744-d147-4790-a701-5f55c0e1969a None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:38.979303 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 374/733] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 19:34:38.980521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc466de-150b-4694-bec2-c2120179b1d0 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:38.989958 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:39.084909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1480744-d147-4790-a701-5f55c0e1969a None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:39.095789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:39.138502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f420ced6-c51c-4c0e-944f-15be17b3ebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:39.140897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420ced6-c51c-4c0e-944f-15be17b3ebbd None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:39.142167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420ced6-c51c-4c0e-944f-15be17b3ebbd None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:39.142669 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420ced6-c51c-4c0e-944f-15be17b3ebbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:39.179257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc466de-150b-4694-bec2-c2120179b1d0 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:39.232617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b72a51-52f4-4dbe-bd26-acc2387d34dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:39.233406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b72a51-52f4-4dbe-bd26-acc2387d34dd None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:39.233683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b72a51-52f4-4dbe-bd26-acc2387d34dd None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:39.233983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b72a51-52f4-4dbe-bd26-acc2387d34dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:39.244269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b72a51-52f4-4dbe-bd26-acc2387d34dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:39.248244 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420ced6-c51c-4c0e-944f-15be17b3ebbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-00b72a51-52f4-4dbe-bd26-acc2387d34dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:39.249463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:39.250587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:39.251424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:39.251424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:39.251424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:39.251424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:39.251424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:39.251424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:39.251424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:39.251424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:39.251424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:39.251424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:39.251424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:34:39.251424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:39.252066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f420ced6-c51c-4c0e-944f-15be17b3ebbd None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:39.255312 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 375/734] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:34:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:39.256049 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b72a51-52f4-4dbe-bd26-acc2387d34dd None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:39.318749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:39.352198 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 360/735] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 19:34:39.353522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5826801-f973-485e-b029-9fa717353cf5 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:39.353827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5826801-f973-485e-b029-9fa717353cf5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:39.354165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5826801-f973-485e-b029-9fa717353cf5 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:39.355332 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 361/736] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:39.356113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5826801-f973-485e-b029-9fa717353cf5 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:39.360860 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b72a51-52f4-4dbe-bd26-acc2387d34dd None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:39.364487 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:39.394753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b1ae6-eab4-43c1-a3f0-c1def685608e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:39.395380 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b1ae6-eab4-43c1-a3f0-c1def685608e None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:39.395659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b1ae6-eab4-43c1-a3f0-c1def685608e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:39.395928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b1ae6-eab4-43c1-a3f0-c1def685608e None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:39.396565 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b1ae6-eab4-43c1-a3f0-c1def685608e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:39.397966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b1ae6-eab4-43c1-a3f0-c1def685608e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:39.457129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5826801-f973-485e-b029-9fa717353cf5 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:39.489064 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 376/737] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 19:34:39 2022] POST /identity/v3/services => generated 240 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:34:39.490003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b1ae6-eab4-43c1-a3f0-c1def685608e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:39.497917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96551e-e985-4c4b-95f0-4df833b31e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:39.497917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96551e-e985-4c4b-95f0-4df833b31e8d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:39.497917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96551e-e985-4c4b-95f0-4df833b31e8d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:39.498267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96551e-e985-4c4b-95f0-4df833b31e8d None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:39.498695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96551e-e985-4c4b-95f0-4df833b31e8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:39.498996 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96551e-e985-4c4b-95f0-4df833b31e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:39.500399 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96551e-e985-4c4b-95f0-4df833b31e8d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:39.511132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 362/738] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:39.512672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96551e-e985-4c4b-95f0-4df833b31e8d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:39.583925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:39.617242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b1ae6-eab4-43c1-a3f0-c1def685608e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:39.631957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96551e-e985-4c4b-95f0-4df833b31e8d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:39.643744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:39.656830 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d337ec2-6f1f-4700-ad6d-a8ad8408a737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:39.657488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d337ec2-6f1f-4700-ad6d-a8ad8408a737 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:39.657767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d337ec2-6f1f-4700-ad6d-a8ad8408a737 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:39.658089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d337ec2-6f1f-4700-ad6d-a8ad8408a737 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:39.669571 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20d0fa-1fb0-4391-b5e1-cfd2116098c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:39.670063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d337ec2-6f1f-4700-ad6d-a8ad8408a737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:39.670235 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20d0fa-1fb0-4391-b5e1-cfd2116098c0 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:39.670507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20d0fa-1fb0-4391-b5e1-cfd2116098c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:39.670780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b20d0fa-1fb0-4391-b5e1-cfd2116098c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-2d337ec2-6f1f-4700-ad6d-a8ad8408a737 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:39.675135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:39.676057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:39.676881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:39.676881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:39.676881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:39.676881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:39.676881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:39.676881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:39.676881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:39.676881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:39.676881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:39.676881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:39.676881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:39.676881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:39.681209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 377/739] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:39.681949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20d0fa-1fb0-4391-b5e1-cfd2116098c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:39.683204 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac4e8-cbf7-40af-8846-cfed89085122 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:39.683204 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac4e8-cbf7-40af-8846-cfed89085122 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:39.683204 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac4e8-cbf7-40af-8846-cfed89085122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:39.683435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b20d0fa-1fb0-4391-b5e1-cfd2116098c0 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-9b20d0fa-1fb0-4391-b5e1-cfd2116098c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:39.701077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:39.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:39.703142 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:39.703142 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:39.703142 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:39.703142 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 363/740] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:39.703397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3d1ac4e8-cbf7-40af-8846-cfed89085122 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:39.703479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20d0fa-1fb0-4391-b5e1-cfd2116098c0 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:39.704335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3d1ac4e8-cbf7-40af-8846-cfed89085122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:39.785904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 378/741] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:39 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 19:34:39.785904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac4e8-cbf7-40af-8846-cfed89085122 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:39.807377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b20d0fa-1fb0-4391-b5e1-cfd2116098c0 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:39.825329 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:39.844933 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8a3e7-33b9-4ff5-b1cf-a89ee9c82145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:39.845921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8a3e7-33b9-4ff5-b1cf-a89ee9c82145 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:39.846276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8a3e7-33b9-4ff5-b1cf-a89ee9c82145 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:39.847264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8a3e7-33b9-4ff5-b1cf-a89ee9c82145 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:39.861725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8a3e7-33b9-4ff5-b1cf-a89ee9c82145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:39.864374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8a3e7-33b9-4ff5-b1cf-a89ee9c82145 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-d6e8a3e7-33b9-4ff5-b1cf-a89ee9c82145 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:39.884576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:39.885527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:39.886476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:39.886476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:39.886476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:39.887892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 364/742] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:39.888656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8a3e7-33b9-4ff5-b1cf-a89ee9c82145 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:39.915796 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac4e8-cbf7-40af-8846-cfed89085122 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:39.933303 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:39.960536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e0d80-0d45-479e-8d99-703e918a3ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:39.961160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e0d80-0d45-479e-8d99-703e918a3ea4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:39.961438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e0d80-0d45-479e-8d99-703e918a3ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:39.961711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e0d80-0d45-479e-8d99-703e918a3ea4 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:39.962441 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e0d80-0d45-479e-8d99-703e918a3ea4 None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:39.962747 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e0d80-0d45-479e-8d99-703e918a3ea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:39.964162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306e0d80-0d45-479e-8d99-703e918a3ea4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:39.977181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 379/743] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:39.979027 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e0d80-0d45-479e-8d99-703e918a3ea4 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:40.026484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8a3e7-33b9-4ff5-b1cf-a89ee9c82145 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:40.063282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815418ea-305d-439e-9b45-f771a0694a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:40.063893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815418ea-305d-439e-9b45-f771a0694a45 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:40.064230 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815418ea-305d-439e-9b45-f771a0694a45 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:40.064568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815418ea-305d-439e-9b45-f771a0694a45 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:40.065266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815418ea-305d-439e-9b45-f771a0694a45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:40.065628 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815418ea-305d-439e-9b45-f771a0694a45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:40.067025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815418ea-305d-439e-9b45-f771a0694a45 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:40.079585 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 365/744] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:40.080656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815418ea-305d-439e-9b45-f771a0694a45 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:40.109326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:40.113338 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e0d80-0d45-479e-8d99-703e918a3ea4 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:40.149274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8fbac-c545-464d-81be-36e0e70887a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:40.149916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8fbac-c545-464d-81be-36e0e70887a9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:40.150199 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8fbac-c545-464d-81be-36e0e70887a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:40.150474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8fbac-c545-464d-81be-36e0e70887a9 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:40.151109 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8fbac-c545-464d-81be-36e0e70887a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:40.151412 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8fbac-c545-464d-81be-36e0e70887a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:40.153443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8fbac-c545-464d-81be-36e0e70887a9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:40.165891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 380/745] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:40.166718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8fbac-c545-464d-81be-36e0e70887a9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:40.188624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815418ea-305d-439e-9b45-f771a0694a45 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:40.204557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:40.228740 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c47b82-79e3-4ebb-b77a-42ab00a651c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:40.229501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c47b82-79e3-4ebb-b77a-42ab00a651c2 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:40.229844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c47b82-79e3-4ebb-b77a-42ab00a651c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:40.230096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c47b82-79e3-4ebb-b77a-42ab00a651c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:40.240754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c47b82-79e3-4ebb-b77a-42ab00a651c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:40.242250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c47b82-79e3-4ebb-b77a-42ab00a651c2 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-29c47b82-79e3-4ebb-b77a-42ab00a651c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:40.257692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:40.259144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:40.260054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:40.260054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:40.260054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:40.260054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 366/746] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:40.260054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c47b82-79e3-4ebb-b77a-42ab00a651c2 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:40.280509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8fbac-c545-464d-81be-36e0e70887a9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:40.317768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad7b9b-7fc1-4fcb-ae11-54be9adbe1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:40.318333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7b9b-7fc1-4fcb-ae11-54be9adbe1ea None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:40.318610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7b9b-7fc1-4fcb-ae11-54be9adbe1ea None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:40.318882 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad7b9b-7fc1-4fcb-ae11-54be9adbe1ea None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:40.319526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad7b9b-7fc1-4fcb-ae11-54be9adbe1ea None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:40.320880 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad7b9b-7fc1-4fcb-ae11-54be9adbe1ea None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:40.343087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:40.361751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c47b82-79e3-4ebb-b77a-42ab00a651c2 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:40.393702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b7286-7b30-4173-9b18-fe609d08424d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:40.394356 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b7286-7b30-4173-9b18-fe609d08424d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:40.394632 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b7286-7b30-4173-9b18-fe609d08424d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:40.394903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b7286-7b30-4173-9b18-fe609d08424d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:40.401903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 381/747] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 19:34:40 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:34:40.401903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad7b9b-7fc1-4fcb-ae11-54be9adbe1ea None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:40.409527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b7286-7b30-4173-9b18-fe609d08424d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:40.409527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b7286-7b30-4173-9b18-fe609d08424d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-831b7286-7b30-4173-9b18-fe609d08424d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:40.422727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:40.423738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:40.425315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:40.425315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:40.425315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:40.425315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:40.425315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:40.425315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:40.425315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:40.425315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:40.425315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 367/748] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:40.425315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b7286-7b30-4173-9b18-fe609d08424d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:40.515897 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:40.537011 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad7b9b-7fc1-4fcb-ae11-54be9adbe1ea None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:40.540642 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b7286-7b30-4173-9b18-fe609d08424d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:40.581713 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efacae-ac69-4df8-91b1-07ee0be5e248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:40.582276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efacae-ac69-4df8-91b1-07ee0be5e248 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:40.582551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efacae-ac69-4df8-91b1-07ee0be5e248 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:40.582819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33efacae-ac69-4df8-91b1-07ee0be5e248 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:40.583744 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efacae-ac69-4df8-91b1-07ee0be5e248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:40.584054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efacae-ac69-4df8-91b1-07ee0be5e248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:40.585589 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33efacae-ac69-4df8-91b1-07ee0be5e248 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:40.596221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:40.600458 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 368/749] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:40.601272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efacae-ac69-4df8-91b1-07ee0be5e248 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:40.609271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773054d7-3867-4e3a-8943-4c973d8a04ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:40.610710 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773054d7-3867-4e3a-8943-4c973d8a04ea None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:40.611162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773054d7-3867-4e3a-8943-4c973d8a04ea None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:40.611597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773054d7-3867-4e3a-8943-4c973d8a04ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:40.623539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773054d7-3867-4e3a-8943-4c973d8a04ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:40.625095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773054d7-3867-4e3a-8943-4c973d8a04ea None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-773054d7-3867-4e3a-8943-4c973d8a04ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:40.641643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:40.642687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:40.643640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:40.643640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:40.643640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:40.643640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:40.643640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:40.643640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:40.643640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:40.643640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:40.643640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:34:40.643640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:40.643640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 382/750] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 19:34:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:40.644247 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773054d7-3867-4e3a-8943-4c973d8a04ea None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:40.728414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33efacae-ac69-4df8-91b1-07ee0be5e248 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:40.759917 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773054d7-3867-4e3a-8943-4c973d8a04ea None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:40.778916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31b829-7412-4343-a840-1643a6034e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:40.779863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31b829-7412-4343-a840-1643a6034e39 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:40.780385 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31b829-7412-4343-a840-1643a6034e39 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:40.780909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31b829-7412-4343-a840-1643a6034e39 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:40.782398 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31b829-7412-4343-a840-1643a6034e39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:40.783188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31b829-7412-4343-a840-1643a6034e39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:40.785494 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31b829-7412-4343-a840-1643a6034e39 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:40.796481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8504acf-de6d-457d-9dfc-c74d9251edea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:40.797164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8504acf-de6d-457d-9dfc-c74d9251edea None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:40.797444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8504acf-de6d-457d-9dfc-c74d9251edea None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:40.797718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8504acf-de6d-457d-9dfc-c74d9251edea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:40.807993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 369/751] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:40.807993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31b829-7412-4343-a840-1643a6034e39 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:40.809757 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8504acf-de6d-457d-9dfc-c74d9251edea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:40.811298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8504acf-de6d-457d-9dfc-c74d9251edea None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-f8504acf-de6d-457d-9dfc-c74d9251edea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:40.823353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:40.824474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:40.825450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:40.825450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:40.825450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:40.825450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:40.825450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:40.825450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:40.825450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:40.825450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:40.825450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 19:34:40.825450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:40.829193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 383/752] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 19:34:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:40.830112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8504acf-de6d-457d-9dfc-c74d9251edea None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:40.846364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:40.902175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:40.959965 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31b829-7412-4343-a840-1643a6034e39 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:40.964375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8504acf-de6d-457d-9dfc-c74d9251edea None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:40.999990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba141d0-9e43-411b-a99e-22193068307e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:41.000619 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba141d0-9e43-411b-a99e-22193068307e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:41.000895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba141d0-9e43-411b-a99e-22193068307e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:41.001164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba141d0-9e43-411b-a99e-22193068307e None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:41.005424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba141d0-9e43-411b-a99e-22193068307e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:41.005424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba141d0-9e43-411b-a99e-22193068307e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:41.005424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba141d0-9e43-411b-a99e-22193068307e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:41.011497 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea149f6-1e3c-4539-9c4a-a88c16c0dbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:41.012682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea149f6-1e3c-4539-9c4a-a88c16c0dbfd None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:41.013191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea149f6-1e3c-4539-9c4a-a88c16c0dbfd None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:41.013738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea149f6-1e3c-4539-9c4a-a88c16c0dbfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:41.014818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea149f6-1e3c-4539-9c4a-a88c16c0dbfd None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=e3d2f1ab1bfd435183e33168400c0b17) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:41.015203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea149f6-1e3c-4539-9c4a-a88c16c0dbfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:41.016700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea149f6-1e3c-4539-9c4a-a88c16c0dbfd None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:41.022774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 384/753] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:40 2022] GET /identity/v3/role_assignments?user.id=e3d2f1ab1bfd435183e33168400c0b17&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:41.023840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea149f6-1e3c-4539-9c4a-a88c16c0dbfd None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:41.024507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 370/754] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 19:34:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:41.038866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba141d0-9e43-411b-a99e-22193068307e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:41.134530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:41.149469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea149f6-1e3c-4539-9c4a-a88c16c0dbfd None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:41.191970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbe143-476a-4e8e-b6ce-667f2a9ba9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:41.192977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbe143-476a-4e8e-b6ce-667f2a9ba9e4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:41.193337 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbe143-476a-4e8e-b6ce-667f2a9ba9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:41.193689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbe143-476a-4e8e-b6ce-667f2a9ba9e4 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:41.194543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cbe143-476a-4e8e-b6ce-667f2a9ba9e4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:41.195200 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cbe143-476a-4e8e-b6ce-667f2a9ba9e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:41.196590 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cbe143-476a-4e8e-b6ce-667f2a9ba9e4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:41.220718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 385/755] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Sep 14 19:34:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:41.223527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbe143-476a-4e8e-b6ce-667f2a9ba9e4 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:41.292953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba141d0-9e43-411b-a99e-22193068307e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:41.326699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:41.337580 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cbe143-476a-4e8e-b6ce-667f2a9ba9e4 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:41.359611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c6b06-ae0d-46f0-94bc-da95ffe97100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:41.360243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c6b06-ae0d-46f0-94bc-da95ffe97100 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:41.360575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c6b06-ae0d-46f0-94bc-da95ffe97100 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:41.360847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c6b06-ae0d-46f0-94bc-da95ffe97100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:41.374215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:41.382427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb3b7d-3117-4327-ba3f-7f774deddb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:41.383304 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb3b7d-3117-4327-ba3f-7f774deddb54 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:41.384811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb3b7d-3117-4327-ba3f-7f774deddb54 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:41.385480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb3b7d-3117-4327-ba3f-7f774deddb54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:41.386453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c6b06-ae0d-46f0-94bc-da95ffe97100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:41.390734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34c6b06-ae0d-46f0-94bc-da95ffe97100 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-b34c6b06-ae0d-46f0-94bc-da95ffe97100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:41.416643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:41.417735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:41.419212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:41.419212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:41.419212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:41.421449 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 371/756] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:41.427404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c6b06-ae0d-46f0-94bc-da95ffe97100 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:41.528961 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb3b7d-3117-4327-ba3f-7f774deddb54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:41.530507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb3b7d-3117-4327-ba3f-7f774deddb54 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:41.581182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c6b06-ae0d-46f0-94bc-da95ffe97100 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:41.623405 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479bd58-32dc-4b49-a479-a29ea1b0e5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:41.624364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479bd58-32dc-4b49-a479-a29ea1b0e5ce None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:41.624849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479bd58-32dc-4b49-a479-a29ea1b0e5ce None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:41.625353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4479bd58-32dc-4b49-a479-a29ea1b0e5ce None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:41.626702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479bd58-32dc-4b49-a479-a29ea1b0e5ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:41.627216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479bd58-32dc-4b49-a479-a29ea1b0e5ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:41.628813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4479bd58-32dc-4b49-a479-a29ea1b0e5ce None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:41.644547 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 372/757] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:41.651559 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479bd58-32dc-4b49-a479-a29ea1b0e5ce None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:41.702267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:41.749052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 386/758] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:41 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 19:34:41.762856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb3b7d-3117-4327-ba3f-7f774deddb54 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:41.806933 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4479bd58-32dc-4b49-a479-a29ea1b0e5ce None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:41.814176 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:41.846661 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720afb9e-e6fc-452f-8665-510710be3da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:41.847291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720afb9e-e6fc-452f-8665-510710be3da4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:41.847572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720afb9e-e6fc-452f-8665-510710be3da4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:41.847847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720afb9e-e6fc-452f-8665-510710be3da4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:41.858634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720afb9e-e6fc-452f-8665-510710be3da4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:41.860176 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720afb9e-e6fc-452f-8665-510710be3da4 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-720afb9e-e6fc-452f-8665-510710be3da4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:41.877135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:41.878069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:41.879389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:41.879389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:41.879389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:41.879389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:41.879389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:41.879389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:41.879389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:41.879389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:41.879389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 373/759] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:41.879877 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74245c48-bfc6-40a8-a021-f298130bcac6 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:41.884559 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74245c48-bfc6-40a8-a021-f298130bcac6 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:41.884883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74245c48-bfc6-40a8-a021-f298130bcac6 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:41.890883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 374/760] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:41.890883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74245c48-bfc6-40a8-a021-f298130bcac6 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:41.905187 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb3b7d-3117-4327-ba3f-7f774deddb54 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:41.942701 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5cb9c3-e057-47a2-b30e-3e2671771780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:41.943339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cb9c3-e057-47a2-b30e-3e2671771780 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:41.943677 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cb9c3-e057-47a2-b30e-3e2671771780 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:41.943956 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5cb9c3-e057-47a2-b30e-3e2671771780 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:41.954688 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5cb9c3-e057-47a2-b30e-3e2671771780 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:41.956722 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5cb9c3-e057-47a2-b30e-3e2671771780 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-9c5cb9c3-e057-47a2-b30e-3e2671771780 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:41.972502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:41.973929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:41.975286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:41.975286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:41.975286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:41.978189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 387/761] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:41.979756 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c41f7-faed-40bc-b0e9-d990c437ec4a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:41.980039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c41f7-faed-40bc-b0e9-d990c437ec4a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:41.980333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c41f7-faed-40bc-b0e9-d990c437ec4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:42.001695 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-867c41f7-faed-40bc-b0e9-d990c437ec4a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:42.002948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-867c41f7-faed-40bc-b0e9-d990c437ec4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:42.016641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:42.029307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74245c48-bfc6-40a8-a021-f298130bcac6 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:42.064650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d064e-0d9e-4b2e-837a-5e57cbd4b178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:42.065284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d064e-0d9e-4b2e-837a-5e57cbd4b178 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:42.065610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d064e-0d9e-4b2e-837a-5e57cbd4b178 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:42.065938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d064e-0d9e-4b2e-837a-5e57cbd4b178 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:42.066632 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d064e-0d9e-4b2e-837a-5e57cbd4b178 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:42.066995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d064e-0d9e-4b2e-837a-5e57cbd4b178 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:42.068750 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d064e-0d9e-4b2e-837a-5e57cbd4b178 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:42.089834 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 375/763] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:42.089834 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d064e-0d9e-4b2e-837a-5e57cbd4b178 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:42.090762 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 388/763] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:34:42.091676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c41f7-faed-40bc-b0e9-d990c437ec4a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:42.123677 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:42.236819 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c41f7-faed-40bc-b0e9-d990c437ec4a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:42.276144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7df6cc-7540-4ff8-b606-f00b8fd6d539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:42.278282 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7df6cc-7540-4ff8-b606-f00b8fd6d539 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:42.278749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7df6cc-7540-4ff8-b606-f00b8fd6d539 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:42.279238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7df6cc-7540-4ff8-b606-f00b8fd6d539 None admin] PATH_INFO: `/v3/projects/6bfc6c1c443849038af35577daf34088/users/2a4284b59f804dc59674d621d87634bc/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:42.280238 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d064e-0d9e-4b2e-837a-5e57cbd4b178 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:42.281540 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7df6cc-7540-4ff8-b606-f00b8fd6d539 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bfc6c1c443849038af35577daf34088, user_id=2a4284b59f804dc59674d621d87634bc, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:42.283352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7df6cc-7540-4ff8-b606-f00b8fd6d539 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:42.317471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b46c88-731e-4bd1-b82b-d223a12db4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:42.318118 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b46c88-731e-4bd1-b82b-d223a12db4fd None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:42.318611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b46c88-731e-4bd1-b82b-d223a12db4fd None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:42.318990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b46c88-731e-4bd1-b82b-d223a12db4fd None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:42.319719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b46c88-731e-4bd1-b82b-d223a12db4fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:42.320095 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b46c88-731e-4bd1-b82b-d223a12db4fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:42.321636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b46c88-731e-4bd1-b82b-d223a12db4fd None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:42.334284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 376/764] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:42.339813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b46c88-731e-4bd1-b82b-d223a12db4fd None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:42.354249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:42.377959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 389/765] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:34:42 2022] PUT /identity/v3/projects/6bfc6c1c443849038af35577daf34088/users/2a4284b59f804dc59674d621d87634bc/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:34:42.472581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b46c88-731e-4bd1-b82b-d223a12db4fd None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:42.476881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7df6cc-7540-4ff8-b606-f00b8fd6d539 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:42.523968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31570515-2afe-4076-bdd2-fd4a89a17017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:42.524534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31570515-2afe-4076-bdd2-fd4a89a17017 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:42.524815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31570515-2afe-4076-bdd2-fd4a89a17017 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:42.525089 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31570515-2afe-4076-bdd2-fd4a89a17017 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:42.536036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31570515-2afe-4076-bdd2-fd4a89a17017 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:42.537505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31570515-2afe-4076-bdd2-fd4a89a17017 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-31570515-2afe-4076-bdd2-fd4a89a17017 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:42.556535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:42.557608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:42.559290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:42.559290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:42.559290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:42.559290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:42.559290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:42.559290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:42.559290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:42.559290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:42.559290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 377/766] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:42.564586 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31570515-2afe-4076-bdd2-fd4a89a17017 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:42.657474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:42.700780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7df6cc-7540-4ff8-b606-f00b8fd6d539 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:42.705161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:42.744849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31570515-2afe-4076-bdd2-fd4a89a17017 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:42.752121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dceacd-cb45-44fe-ac5d-732d5a95419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:42.753182 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dceacd-cb45-44fe-ac5d-732d5a95419f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:42.753462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dceacd-cb45-44fe-ac5d-732d5a95419f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:42.753735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dceacd-cb45-44fe-ac5d-732d5a95419f None admin] PATH_INFO: `/v3/services/network` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:42.754706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dceacd-cb45-44fe-ac5d-732d5a95419f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:42.756547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dceacd-cb45-44fe-ac5d-732d5a95419f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-e1dceacd-cb45-44fe-ac5d-732d5a95419f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:42.768122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:42.769129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:42.770434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 390/767] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:42 2022] GET /identity/v3/services/network => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:42.771233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66723b0-3679-477b-8253-6bb8acc75885 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:42.771590 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66723b0-3679-477b-8253-6bb8acc75885 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:42.771753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66723b0-3679-477b-8253-6bb8acc75885 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:42.773631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 391/768] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:42.778992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66723b0-3679-477b-8253-6bb8acc75885 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:42.783275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7943faf-c3dd-45a3-b933-20f8425d06a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:42.783891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7943faf-c3dd-45a3-b933-20f8425d06a7 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:42.784164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7943faf-c3dd-45a3-b933-20f8425d06a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:42.784490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7943faf-c3dd-45a3-b933-20f8425d06a7 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:42.785178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7943faf-c3dd-45a3-b933-20f8425d06a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:42.785476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7943faf-c3dd-45a3-b933-20f8425d06a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:42.786946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7943faf-c3dd-45a3-b933-20f8425d06a7 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:42.801528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 378/769] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:42.803026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d485e0-4bd4-4269-b33d-7366d3c6e03c None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:42.803310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d485e0-4bd4-4269-b33d-7366d3c6e03c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:42.803579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d485e0-4bd4-4269-b33d-7366d3c6e03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:42.822466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-30d485e0-4bd4-4269-b33d-7366d3c6e03c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:42.823785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-30d485e0-4bd4-4269-b33d-7366d3c6e03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:42.890032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66723b0-3679-477b-8253-6bb8acc75885 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:42.899341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 379/770] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:34:42.900064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d485e0-4bd4-4269-b33d-7366d3c6e03c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:42.967447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca05df1-4e03-4598-96f4-e923cc6f304d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:42.968067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca05df1-4e03-4598-96f4-e923cc6f304d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:42.968345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca05df1-4e03-4598-96f4-e923cc6f304d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:42.968616 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca05df1-4e03-4598-96f4-e923cc6f304d None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:42.969243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca05df1-4e03-4598-96f4-e923cc6f304d None admin] RBAC: Adding query filter params (name=network) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:42.969540 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca05df1-4e03-4598-96f4-e923cc6f304d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:42.971525 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca05df1-4e03-4598-96f4-e923cc6f304d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:42.984526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 392/771] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 19:34:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:42.990271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:42.990889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca05df1-4e03-4598-96f4-e923cc6f304d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:42.996817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:43.042648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d485e0-4bd4-4269-b33d-7366d3c6e03c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:43.084668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd55423-80c7-4f1a-83e9-27489d728a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:43.085271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd55423-80c7-4f1a-83e9-27489d728a68 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:43.085545 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd55423-80c7-4f1a-83e9-27489d728a68 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:43.085831 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd55423-80c7-4f1a-83e9-27489d728a68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:43.086549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd55423-80c7-4f1a-83e9-27489d728a68 None admin] RBAC: Adding query filter params (role.id=928c1b8875334543ab431e4f7e2acee5, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=7d5605f645a74b1b842733dda256193c) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:43.087260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd55423-80c7-4f1a-83e9-27489d728a68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:43.089003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd55423-80c7-4f1a-83e9-27489d728a68 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:43.095787 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 380/772] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:42 2022] GET /identity/v3/role_assignments?user.id=7d5605f645a74b1b842733dda256193c&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=928c1b8875334543ab431e4f7e2acee5 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:43.151568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca05df1-4e03-4598-96f4-e923cc6f304d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:43.187903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e4202-43fe-4403-98e6-d9ade04b2910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:43.188748 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e4202-43fe-4403-98e6-d9ade04b2910 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:43.189028 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e4202-43fe-4403-98e6-d9ade04b2910 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:43.189340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e4202-43fe-4403-98e6-d9ade04b2910 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:43.190018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e4202-43fe-4403-98e6-d9ade04b2910 None admin] RBAC: Adding query filter params (type=network) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:43.190321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e4202-43fe-4403-98e6-d9ade04b2910 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:43.191731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47e4202-43fe-4403-98e6-d9ade04b2910 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:43.197985 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 393/773] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 19:34:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:43.206427 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd55423-80c7-4f1a-83e9-27489d728a68 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:43.240205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e4202-43fe-4403-98e6-d9ade04b2910 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:43.360739 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:43.399937 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd55423-80c7-4f1a-83e9-27489d728a68 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:43.423200 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e4202-43fe-4403-98e6-d9ade04b2910 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:43.433027 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:43.443374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255691d7-2fce-42a1-a62d-aa4c6edbf37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:43.444025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255691d7-2fce-42a1-a62d-aa4c6edbf37f None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:43.444300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255691d7-2fce-42a1-a62d-aa4c6edbf37f None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:43.444565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255691d7-2fce-42a1-a62d-aa4c6edbf37f None admin] PATH_INFO: `/v3/endpoints` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:43.445257 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255691d7-2fce-42a1-a62d-aa4c6edbf37f None admin] RBAC: Adding query filter params (interface=public, service_id=2b970fe751ec468abddc12257195011f, region_id=RegionOne) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:43.445557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255691d7-2fce-42a1-a62d-aa4c6edbf37f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:43.447057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255691d7-2fce-42a1-a62d-aa4c6edbf37f None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:43.461279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd1fb9-24a2-47af-979a-116249408ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:43.462273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd1fb9-24a2-47af-979a-116249408ad4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:43.462849 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd1fb9-24a2-47af-979a-116249408ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:43.463032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 381/774] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 19:34:43 2022] GET /identity/v3/endpoints?service_id=2b970fe751ec468abddc12257195011f&interface=public®ion_id=RegionOne => generated 194 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:43.463506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd1fb9-24a2-47af-979a-116249408ad4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:43.464333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd1fb9-24a2-47af-979a-116249408ad4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:43.466561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd1fb9-24a2-47af-979a-116249408ad4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:43.469245 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255691d7-2fce-42a1-a62d-aa4c6edbf37f None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-91fd1fb9-24a2-47af-979a-116249408ad4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:43.476296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:43.477300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:43.478178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:43.478178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:43.478178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:43.478178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:43.478178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:43.478178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:43.478178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:43.478178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:43.478178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:43.478178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:43.478178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:34:43.478178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:43.478178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 19:34:43.478178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:43.480888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 394/775] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Sep 14 19:34:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:43.486809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd1fb9-24a2-47af-979a-116249408ad4 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:43.594413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd1fb9-24a2-47af-979a-116249408ad4 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:43.595940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255691d7-2fce-42a1-a62d-aa4c6edbf37f None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:43.625154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:43.632005 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198f82d-5bf7-430b-b50f-0cc85ef95918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:43.632846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a08b8-fca2-4efa-8825-2731d0f1c1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:43.646127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198f82d-5bf7-430b-b50f-0cc85ef95918 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:43.646127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198f82d-5bf7-430b-b50f-0cc85ef95918 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:43.646322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a08b8-fca2-4efa-8825-2731d0f1c1f1 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:43.646396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198f82d-5bf7-430b-b50f-0cc85ef95918 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:43.646466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a08b8-fca2-4efa-8825-2731d0f1c1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:43.646466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a08b8-fca2-4efa-8825-2731d0f1c1f1 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:43.646639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198f82d-5bf7-430b-b50f-0cc85ef95918 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:43.646639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198f82d-5bf7-430b-b50f-0cc85ef95918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:43.646766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a08b8-fca2-4efa-8825-2731d0f1c1f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:43.646766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a08b8-fca2-4efa-8825-2731d0f1c1f1 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:43.646887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198f82d-5bf7-430b-b50f-0cc85ef95918 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:43.646963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 382/777] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 19:34:43 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:43.646963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55373fc-9f1f-4835-bcf9-c79dd102369c None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:43.646963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55373fc-9f1f-4835-bcf9-c79dd102369c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:43.646963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55373fc-9f1f-4835-bcf9-c79dd102369c None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:43.646963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 383/778] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:43.647248 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 395/778] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Sep 14 19:34:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:43.648801 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55373fc-9f1f-4835-bcf9-c79dd102369c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:43.660039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b314d-1d0d-4af6-b00e-0f52acdc86f7 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:43.660686 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b314d-1d0d-4af6-b00e-0f52acdc86f7 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:43.661172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b314d-1d0d-4af6-b00e-0f52acdc86f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:43.705922 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3e1b314d-1d0d-4af6-b00e-0f52acdc86f7 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:43.707315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3e1b314d-1d0d-4af6-b00e-0f52acdc86f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:43.738794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:43.830516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 396/779] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:34:43.832664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55373fc-9f1f-4835-bcf9-c79dd102369c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:43.876961 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f6c21-5066-4022-9a14-d22bacad1697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:43.877574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f6c21-5066-4022-9a14-d22bacad1697 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:43.877870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f6c21-5066-4022-9a14-d22bacad1697 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:43.878264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f6c21-5066-4022-9a14-d22bacad1697 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:43.878890 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f6c21-5066-4022-9a14-d22bacad1697 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:43.879190 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f6c21-5066-4022-9a14-d22bacad1697 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:43.880541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f6c21-5066-4022-9a14-d22bacad1697 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:43.887273 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 384/780] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Sep 14 19:34:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:43.892994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b314d-1d0d-4af6-b00e-0f52acdc86f7 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:43.994215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b314d-1d0d-4af6-b00e-0f52acdc86f7 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:44.018419 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:44.029663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d9367-63a7-478e-9fbe-c29fc05b330b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:44.030297 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d9367-63a7-478e-9fbe-c29fc05b330b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:44.030573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d9367-63a7-478e-9fbe-c29fc05b330b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:44.030843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790d9367-63a7-478e-9fbe-c29fc05b330b None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:44.031532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d9367-63a7-478e-9fbe-c29fc05b330b None admin] RBAC: Adding query filter params (interface=public, service_id=be49055b62d9433ab3e8c044801a241c, region_id=RegionOne) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:44.031833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d9367-63a7-478e-9fbe-c29fc05b330b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:44.033245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790d9367-63a7-478e-9fbe-c29fc05b330b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:44.040955 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 397/781] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 19:34:43 2022] GET /identity/v3/endpoints?service_id=be49055b62d9433ab3e8c044801a241c&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:44.047101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f6c21-5066-4022-9a14-d22bacad1697 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:44.169724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d9367-63a7-478e-9fbe-c29fc05b330b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:44.171103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f6c21-5066-4022-9a14-d22bacad1697 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:44.190162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:44.211221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4fe55-2b60-42ff-922a-08c3c1d497b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:44.211873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4fe55-2b60-42ff-922a-08c3c1d497b6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:44.212147 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4fe55-2b60-42ff-922a-08c3c1d497b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:44.212461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4fe55-2b60-42ff-922a-08c3c1d497b6 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:44.213058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4fe55-2b60-42ff-922a-08c3c1d497b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:44.214594 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4fe55-2b60-42ff-922a-08c3c1d497b6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:44.221029 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 385/782] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 19:34:44 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:44.359423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790d9367-63a7-478e-9fbe-c29fc05b330b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:44.411982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f7b6a-dc90-4c09-8169-7d4ef113f0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:44.412645 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f7b6a-dc90-4c09-8169-7d4ef113f0b1 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:44.412925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f7b6a-dc90-4c09-8169-7d4ef113f0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:44.413197 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2f7b6a-dc90-4c09-8169-7d4ef113f0b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:44.431131 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2f7b6a-dc90-4c09-8169-7d4ef113f0b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-9f2f7b6a-dc90-4c09-8169-7d4ef113f0b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:44.436405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:44.437315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:44.438191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:44.438191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:44.438191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:44.438191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:44.438191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:44.438191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:44.438191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:44.438191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:44.438191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:44.438191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:44.438191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:34:44.438191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:44.438191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 398/783] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:34:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:44.444049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4fe55-2b60-42ff-922a-08c3c1d497b6 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:44.555529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4fe55-2b60-42ff-922a-08c3c1d497b6 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:44.591563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad8bda-a517-4ca2-b43d-c37273c0e43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:44.592412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad8bda-a517-4ca2-b43d-c37273c0e43e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:44.592828 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad8bda-a517-4ca2-b43d-c37273c0e43e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:44.593235 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad8bda-a517-4ca2-b43d-c37273c0e43e None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:44.594080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad8bda-a517-4ca2-b43d-c37273c0e43e None admin] RBAC: Adding query filter params (name=admin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:44.594520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad8bda-a517-4ca2-b43d-c37273c0e43e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:44.596021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad8bda-a517-4ca2-b43d-c37273c0e43e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:44.606803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 386/784] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:44.613990 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f7b6a-dc90-4c09-8169-7d4ef113f0b1 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:44.708554 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:44.766469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2f7b6a-dc90-4c09-8169-7d4ef113f0b1 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:44.801225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa73510-de64-498a-b2e9-b6a63f3a2034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:44.801940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa73510-de64-498a-b2e9-b6a63f3a2034 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:44.802218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa73510-de64-498a-b2e9-b6a63f3a2034 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:44.802485 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa73510-de64-498a-b2e9-b6a63f3a2034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:44.813837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa73510-de64-498a-b2e9-b6a63f3a2034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:44.815290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa73510-de64-498a-b2e9-b6a63f3a2034 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-7aa73510-de64-498a-b2e9-b6a63f3a2034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:44.829694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:44.830679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:44.831552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:44.831552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:44.831552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:44.831552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 399/785] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:44.839113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad8bda-a517-4ca2-b43d-c37273c0e43e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:44.911647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:44.986110 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad8bda-a517-4ca2-b43d-c37273c0e43e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:45.028531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bdd243-2ad8-4c89-88c3-3e06c60b2849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:45.029140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdd243-2ad8-4c89-88c3-3e06c60b2849 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:45.029413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdd243-2ad8-4c89-88c3-3e06c60b2849 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:45.029679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bdd243-2ad8-4c89-88c3-3e06c60b2849 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:45.030344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdd243-2ad8-4c89-88c3-3e06c60b2849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:45.030641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdd243-2ad8-4c89-88c3-3e06c60b2849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:45.031995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bdd243-2ad8-4c89-88c3-3e06c60b2849 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:45.052118 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 387/786] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:45.057637 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa73510-de64-498a-b2e9-b6a63f3a2034 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:45.060107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238148e-8f42-4405-aa7a-0bd4cd4303c0 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:45.060391 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238148e-8f42-4405-aa7a-0bd4cd4303c0 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:45.060666 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c238148e-8f42-4405-aa7a-0bd4cd4303c0 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:45.062265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 388/787] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:34:45.073254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fa9a2-9283-46ac-9cac-e676738e9670 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:45.073561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fa9a2-9283-46ac-9cac-e676738e9670 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:45.073788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fa9a2-9283-46ac-9cac-e676738e9670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:45.092740 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b77fa9a2-9283-46ac-9cac-e676738e9670 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:45.094075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b77fa9a2-9283-46ac-9cac-e676738e9670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:45.187178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa73510-de64-498a-b2e9-b6a63f3a2034 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:45.190144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:45.212382 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 389/788] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:34:45.246573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8a9c7-0d64-4655-bff1-a76cd764f1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:45.247567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8a9c7-0d64-4655-bff1-a76cd764f1f2 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:45.248138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8a9c7-0d64-4655-bff1-a76cd764f1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:45.248610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8a9c7-0d64-4655-bff1-a76cd764f1f2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:45.262834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8a9c7-0d64-4655-bff1-a76cd764f1f2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:45.264617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd8a9c7-0d64-4655-bff1-a76cd764f1f2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-0bd8a9c7-0d64-4655-bff1-a76cd764f1f2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:45.284711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:45.286313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:45.287329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:45.287329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:45.287329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:45.287329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:45.287329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:45.287329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:45.287329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:45.287329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:45.287329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 19:34:45.287329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:45.289931 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 400/789] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:34:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:45.298014 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fa9a2-9283-46ac-9cac-e676738e9670 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:45.342462 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:45.432498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77fa9a2-9283-46ac-9cac-e676738e9670 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:45.478836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bee847-6487-4430-bf7d-8f505c354c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:45.479463 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bee847-6487-4430-bf7d-8f505c354c8e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:45.479708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bee847-6487-4430-bf7d-8f505c354c8e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:45.479978 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bee847-6487-4430-bf7d-8f505c354c8e None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:45.480788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bee847-6487-4430-bf7d-8f505c354c8e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:45.481093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bee847-6487-4430-bf7d-8f505c354c8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:45.482536 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bee847-6487-4430-bf7d-8f505c354c8e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:45.500997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 390/790] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Sep 14 19:34:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:45.506704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8a9c7-0d64-4655-bff1-a76cd764f1f2 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:45.691206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bee847-6487-4430-bf7d-8f505c354c8e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:45.781661 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8a9c7-0d64-4655-bff1-a76cd764f1f2 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:45.815641 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674c95f-8c0f-4fcb-88cf-461768a50c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:45.816268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674c95f-8c0f-4fcb-88cf-461768a50c77 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:45.816540 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674c95f-8c0f-4fcb-88cf-461768a50c77 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:45.816807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674c95f-8c0f-4fcb-88cf-461768a50c77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:45.821325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:45.828410 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674c95f-8c0f-4fcb-88cf-461768a50c77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:45.829855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674c95f-8c0f-4fcb-88cf-461768a50c77 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:45.834157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bee847-6487-4430-bf7d-8f505c354c8e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-8674c95f-8c0f-4fcb-88cf-461768a50c77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:45.844360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:45.845295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:45.846234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:45.846234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:45.846234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:45.846234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 401/791] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:45.849833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca61c0-c34f-4cc2-b0a8-9fdf03a66771 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:45.849833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca61c0-c34f-4cc2-b0a8-9fdf03a66771 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:45.849833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeca61c0-c34f-4cc2-b0a8-9fdf03a66771 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:45.849833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 402/792] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:45.853661 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca61c0-c34f-4cc2-b0a8-9fdf03a66771 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:45.873497 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:45.891104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c03eaa-47ba-4cb1-9e50-d4824d3394f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:45.892601 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c03eaa-47ba-4cb1-9e50-d4824d3394f4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:45.893035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c03eaa-47ba-4cb1-9e50-d4824d3394f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:45.893547 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c03eaa-47ba-4cb1-9e50-d4824d3394f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:45.907315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c03eaa-47ba-4cb1-9e50-d4824d3394f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-e3c03eaa-47ba-4cb1-9e50-d4824d3394f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:45.913747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:45.915004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:45.916066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:45.916066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:45.916066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:45.916066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:45.916066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:45.916066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:45.916066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:45.916066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:45.916066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:45.916066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:45.916066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:34:45.916066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:45.924946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 391/793] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:34:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:45.926434 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7780-c157-481b-a4b9-7c6301da9311 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:45.926921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7780-c157-481b-a4b9-7c6301da9311 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:45.927391 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7780-c157-481b-a4b9-7c6301da9311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:45.946612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-4ffb7780-c157-481b-a4b9-7c6301da9311 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:45.948126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4ffb7780-c157-481b-a4b9-7c6301da9311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:45.971083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeca61c0-c34f-4cc2-b0a8-9fdf03a66771 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:46.007330 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63686635-723d-4684-ab06-04e4c93344ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:46.008039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686635-723d-4684-ab06-04e4c93344ea None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:46.008378 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686635-723d-4684-ab06-04e4c93344ea None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:46.008712 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686635-723d-4684-ab06-04e4c93344ea None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:46.009412 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63686635-723d-4684-ab06-04e4c93344ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:46.009887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63686635-723d-4684-ab06-04e4c93344ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:46.011321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63686635-723d-4684-ab06-04e4c93344ea None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:46.022049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 392/794] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:34:46.023298 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb7780-c157-481b-a4b9-7c6301da9311 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:46.031811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 403/795] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:46.036760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63686635-723d-4684-ab06-04e4c93344ea None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:46.110233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:46.133969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb7780-c157-481b-a4b9-7c6301da9311 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:46.181240 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54a78f-069f-4cf9-8036-ae78e8ca19c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:46.182269 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54a78f-069f-4cf9-8036-ae78e8ca19c4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:46.182755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54a78f-069f-4cf9-8036-ae78e8ca19c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:46.183278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54a78f-069f-4cf9-8036-ae78e8ca19c4 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:46.184692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54a78f-069f-4cf9-8036-ae78e8ca19c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:46.185436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54a78f-069f-4cf9-8036-ae78e8ca19c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:46.189654 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54a78f-069f-4cf9-8036-ae78e8ca19c4 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:46.196784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:46.205750 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 393/796] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:46.211568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54a78f-069f-4cf9-8036-ae78e8ca19c4 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:46.248006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63686635-723d-4684-ab06-04e4c93344ea None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:46.297302 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d01758-b6c1-4a62-b636-5ba8a336efc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:46.297984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d01758-b6c1-4a62-b636-5ba8a336efc5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:46.298407 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d01758-b6c1-4a62-b636-5ba8a336efc5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:46.298681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d01758-b6c1-4a62-b636-5ba8a336efc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:46.306413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d01758-b6c1-4a62-b636-5ba8a336efc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:46.307899 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d01758-b6c1-4a62-b636-5ba8a336efc5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-52d01758-b6c1-4a62-b636-5ba8a336efc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:46.328127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:46.329069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:46.329993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:46.329993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:46.329993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:46.329993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:46.329993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:46.329993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:46.329993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:46.329993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:46.331416 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 404/797] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:46.338255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635d6ed-3052-4037-9c88-8256fe530470 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:46.338653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635d6ed-3052-4037-9c88-8256fe530470 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:46.339027 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7635d6ed-3052-4037-9c88-8256fe530470 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:46.342020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 405/798] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:46.342751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635d6ed-3052-4037-9c88-8256fe530470 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:46.367682 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54a78f-069f-4cf9-8036-ae78e8ca19c4 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:46.405341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b72e39-43e3-4a5b-bdb2-a3f95ff96a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:46.406022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b72e39-43e3-4a5b-bdb2-a3f95ff96a47 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:46.406299 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b72e39-43e3-4a5b-bdb2-a3f95ff96a47 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:46.406571 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b72e39-43e3-4a5b-bdb2-a3f95ff96a47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:46.417397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b72e39-43e3-4a5b-bdb2-a3f95ff96a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:46.418885 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b72e39-43e3-4a5b-bdb2-a3f95ff96a47 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:46.420592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-01b72e39-43e3-4a5b-bdb2-a3f95ff96a47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:46.432844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:46.435007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:46.435892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:46.435892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:46.435892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:46.435892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 394/799] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:46.435892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c69e97-4a32-4b71-a0aa-43a65d7a21f5 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:46.435892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c69e97-4a32-4b71-a0aa-43a65d7a21f5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:46.435892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c69e97-4a32-4b71-a0aa-43a65d7a21f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:46.456663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-79c69e97-4a32-4b71-a0aa-43a65d7a21f5 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:46.458061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-79c69e97-4a32-4b71-a0aa-43a65d7a21f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:46.467001 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7635d6ed-3052-4037-9c88-8256fe530470 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:46.495035 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:46.499854 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7750a2-e87a-4e37-8899-839b45971b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:46.500456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7750a2-e87a-4e37-8899-839b45971b29 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:46.500726 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7750a2-e87a-4e37-8899-839b45971b29 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:46.500997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7750a2-e87a-4e37-8899-839b45971b29 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:46.501660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7750a2-e87a-4e37-8899-839b45971b29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:46.501991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7750a2-e87a-4e37-8899-839b45971b29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:46.503409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7750a2-e87a-4e37-8899-839b45971b29 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:46.516620 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 406/800] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:46.517199 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7750a2-e87a-4e37-8899-839b45971b29 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:46.545865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 395/801] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:34:46.545865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c69e97-4a32-4b71-a0aa-43a65d7a21f5 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:46.639165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7750a2-e87a-4e37-8899-839b45971b29 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:46.679850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020294b-f629-4193-8393-a85d703d1d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:46.680539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c69e97-4a32-4b71-a0aa-43a65d7a21f5 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:46.680665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020294b-f629-4193-8393-a85d703d1d3f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:46.680925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020294b-f629-4193-8393-a85d703d1d3f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:46.681268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020294b-f629-4193-8393-a85d703d1d3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:46.694671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020294b-f629-4193-8393-a85d703d1d3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-8020294b-f629-4193-8393-a85d703d1d3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:46.699832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:46.700760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:46.701608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:46.701608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:46.701608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:46.701608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:46.701608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:46.701608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:46.701608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:46.701608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:46.701608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:46.701608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:46.701608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:46.701608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:46.705954 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 407/802] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:46.706787 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020294b-f629-4193-8393-a85d703d1d3f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:46.710158 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:46.725719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5a90b-ab76-45af-95d0-aec7e443c4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:46.726797 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5a90b-ab76-45af-95d0-aec7e443c4f5 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:46.727232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5a90b-ab76-45af-95d0-aec7e443c4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:46.728003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d5a90b-ab76-45af-95d0-aec7e443c4f5 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:46.728785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5a90b-ab76-45af-95d0-aec7e443c4f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:46.729252 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5a90b-ab76-45af-95d0-aec7e443c4f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:46.731894 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d5a90b-ab76-45af-95d0-aec7e443c4f5 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:46.747924 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 396/803] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:46.748990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5a90b-ab76-45af-95d0-aec7e443c4f5 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:46.764753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:46.814088 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020294b-f629-4193-8393-a85d703d1d3f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:46.849982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a28927-3e08-4829-abec-50e3abba12bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:46.850651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a28927-3e08-4829-abec-50e3abba12bd None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:46.850925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a28927-3e08-4829-abec-50e3abba12bd None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:46.851215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a28927-3e08-4829-abec-50e3abba12bd None admin] PATH_INFO: `/v3/projects/6bfc6c1c443849038af35577daf34088/users/e3d2f1ab1bfd435183e33168400c0b17/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:46.853114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a28927-3e08-4829-abec-50e3abba12bd None admin] RBAC: Authorizing `identity:create_grant(project_id=6bfc6c1c443849038af35577daf34088, user_id=e3d2f1ab1bfd435183e33168400c0b17, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:46.857085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a28927-3e08-4829-abec-50e3abba12bd None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:46.880670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d5a90b-ab76-45af-95d0-aec7e443c4f5 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:46.914073 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f443a17-3cb8-4cd7-9055-f39458695109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:46.914605 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f443a17-3cb8-4cd7-9055-f39458695109 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:46.914880 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f443a17-3cb8-4cd7-9055-f39458695109 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:46.915149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f443a17-3cb8-4cd7-9055-f39458695109 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:46.915797 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f443a17-3cb8-4cd7-9055-f39458695109 None admin] RBAC: Adding query filter params (name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:46.916103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f443a17-3cb8-4cd7-9055-f39458695109 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:46.917504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f443a17-3cb8-4cd7-9055-f39458695109 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:46.929850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 408/804] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:34:46 2022] PUT /identity/v3/projects/6bfc6c1c443849038af35577daf34088/users/e3d2f1ab1bfd435183e33168400c0b17/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:34:46.929850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a28927-3e08-4829-abec-50e3abba12bd None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:46.930102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 397/805] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:46.930102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f443a17-3cb8-4cd7-9055-f39458695109 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:46.990968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:47.061404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f443a17-3cb8-4cd7-9055-f39458695109 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:47.080179 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a28927-3e08-4829-abec-50e3abba12bd None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:47.097355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:47.106718 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85d843-fa30-476e-a359-4e3e15445391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:47.107163 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85d843-fa30-476e-a359-4e3e15445391 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:47.107437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85d843-fa30-476e-a359-4e3e15445391 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:47.107703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd85d843-fa30-476e-a359-4e3e15445391 None admin] PATH_INFO: `/v3/services/network` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:47.108285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85d843-fa30-476e-a359-4e3e15445391 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:47.109713 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd85d843-fa30-476e-a359-4e3e15445391 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-bd85d843-fa30-476e-a359-4e3e15445391 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:47.120493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:47.121679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:47.122565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 398/806] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:46 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:47.123370 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85d843-fa30-476e-a359-4e3e15445391 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:47.132296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ef4e0-9247-4a1a-b152-d46dfcbc33d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:47.132938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ef4e0-9247-4a1a-b152-d46dfcbc33d5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:47.133213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ef4e0-9247-4a1a-b152-d46dfcbc33d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:47.133481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ef4e0-9247-4a1a-b152-d46dfcbc33d5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:47.143004 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ef4e0-9247-4a1a-b152-d46dfcbc33d5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:47.144897 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ef4e0-9247-4a1a-b152-d46dfcbc33d5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-113ef4e0-9247-4a1a-b152-d46dfcbc33d5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:47.157135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:47.158358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:47.159720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:47.159720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:47.159720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:47.159720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:47.159720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:47.159720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:47.159720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:47.159720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:47.159720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:34:47.159720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:47.159720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 409/807] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 19:34:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:47.162831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0795e65-2ca9-4ab8-b855-7289c08b55fb None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:47.163274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0795e65-2ca9-4ab8-b855-7289c08b55fb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:47.163767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0795e65-2ca9-4ab8-b855-7289c08b55fb None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:47.169975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 410/808] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:47.170740 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0795e65-2ca9-4ab8-b855-7289c08b55fb None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:47.235649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd85d843-fa30-476e-a359-4e3e15445391 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:47.246068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:47.270833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4e2f3-4c65-4a49-98c2-7174cd0e2444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:47.271453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4e2f3-4c65-4a49-98c2-7174cd0e2444 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:47.271730 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4e2f3-4c65-4a49-98c2-7174cd0e2444 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:47.271996 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4e2f3-4c65-4a49-98c2-7174cd0e2444 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:47.276054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0795e65-2ca9-4ab8-b855-7289c08b55fb None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:47.286166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4e2f3-4c65-4a49-98c2-7174cd0e2444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:47.288016 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b4e2f3-4c65-4a49-98c2-7174cd0e2444 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-e2b4e2f3-4c65-4a49-98c2-7174cd0e2444 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:47.302952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:47.303947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:47.304902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:47.304902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:47.304902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:47.304902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 399/809] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:47.304902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4e2f3-4c65-4a49-98c2-7174cd0e2444 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:47.318110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be112e-6f50-4ec9-9e7f-8a7f76a7beae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:47.318751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be112e-6f50-4ec9-9e7f-8a7f76a7beae None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:47.319029 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be112e-6f50-4ec9-9e7f-8a7f76a7beae None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:47.319299 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be112e-6f50-4ec9-9e7f-8a7f76a7beae None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:47.319925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be112e-6f50-4ec9-9e7f-8a7f76a7beae None admin] RBAC: Adding query filter params (name=network) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:47.320227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be112e-6f50-4ec9-9e7f-8a7f76a7beae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:47.321581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be112e-6f50-4ec9-9e7f-8a7f76a7beae None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:47.331544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 411/810] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 19:34:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:47.331544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db27627-3932-40c2-b35e-f3e5812266eb None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:47.331544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db27627-3932-40c2-b35e-f3e5812266eb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:47.331544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db27627-3932-40c2-b35e-f3e5812266eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:47.347852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2db27627-3932-40c2-b35e-f3e5812266eb None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:47.349037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2db27627-3932-40c2-b35e-f3e5812266eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:47.379720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:47.415194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4e2f3-4c65-4a49-98c2-7174cd0e2444 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:47.421967 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 412/811] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:34:47.423083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db27627-3932-40c2-b35e-f3e5812266eb None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:47.453092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c062a1-505b-45b7-a914-ef3fdf0c2ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:47.454053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c062a1-505b-45b7-a914-ef3fdf0c2ec4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:47.454511 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c062a1-505b-45b7-a914-ef3fdf0c2ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:47.454978 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c062a1-505b-45b7-a914-ef3fdf0c2ec4 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:47.455877 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c062a1-505b-45b7-a914-ef3fdf0c2ec4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:47.456359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c062a1-505b-45b7-a914-ef3fdf0c2ec4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:47.457994 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c062a1-505b-45b7-a914-ef3fdf0c2ec4 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:47.476461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 400/812] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 19:34:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:47.477529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c062a1-505b-45b7-a914-ef3fdf0c2ec4 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:47.500366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:47.557494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db27627-3932-40c2-b35e-f3e5812266eb None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:47.581320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c062a1-505b-45b7-a914-ef3fdf0c2ec4 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:47.607448 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172cbe5-e23e-4841-a9bd-37e6c65043ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:47.608061 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172cbe5-e23e-4841-a9bd-37e6c65043ee None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:47.608339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172cbe5-e23e-4841-a9bd-37e6c65043ee None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:47.608609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172cbe5-e23e-4841-a9bd-37e6c65043ee None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:47.609239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172cbe5-e23e-4841-a9bd-37e6c65043ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:47.609536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172cbe5-e23e-4841-a9bd-37e6c65043ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:47.612476 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172cbe5-e23e-4841-a9bd-37e6c65043ee None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:47.629446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 413/813] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:47.631073 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172cbe5-e23e-4841-a9bd-37e6c65043ee None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:47.635022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654da6a4-4f96-4e2f-a019-5fb7e2d0c8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:47.635655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654da6a4-4f96-4e2f-a019-5fb7e2d0c8d8 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:47.636927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654da6a4-4f96-4e2f-a019-5fb7e2d0c8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:47.637213 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654da6a4-4f96-4e2f-a019-5fb7e2d0c8d8 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:47.637866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654da6a4-4f96-4e2f-a019-5fb7e2d0c8d8 None admin] RBAC: Adding query filter params (type=network) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:47.638178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654da6a4-4f96-4e2f-a019-5fb7e2d0c8d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:47.639544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654da6a4-4f96-4e2f-a019-5fb7e2d0c8d8 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:47.649468 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 401/814] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 19:34:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:47.651979 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654da6a4-4f96-4e2f-a019-5fb7e2d0c8d8 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:47.669992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:47.748927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172cbe5-e23e-4841-a9bd-37e6c65043ee None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:47.753411 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:47.770032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654da6a4-4f96-4e2f-a019-5fb7e2d0c8d8 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:47.783492 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad90f3-9a03-4592-8bff-116eeec4cc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:47.784095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad90f3-9a03-4592-8bff-116eeec4cc3c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:47.784379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad90f3-9a03-4592-8bff-116eeec4cc3c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:47.784650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad90f3-9a03-4592-8bff-116eeec4cc3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:47.805929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b1d41-937b-417c-a1c3-08fe46ba8687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:47.806573 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b1d41-937b-417c-a1c3-08fe46ba8687 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:47.806836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b1d41-937b-417c-a1c3-08fe46ba8687 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:47.807108 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b1d41-937b-417c-a1c3-08fe46ba8687 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:47.815719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b1d41-937b-417c-a1c3-08fe46ba8687 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:47.817188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b1d41-937b-417c-a1c3-08fe46ba8687 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-9c4b1d41-937b-417c-a1c3-08fe46ba8687 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:47.830720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:47.831768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:47.832690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:47.832690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:47.832690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:47.832690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:47.832690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:47.832690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:47.832690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:47.832690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:47.832690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 19:34:47.832690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:47.832690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 402/815] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 19:34:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:47.833287 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b1d41-937b-417c-a1c3-08fe46ba8687 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:47.866287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad90f3-9a03-4592-8bff-116eeec4cc3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:47.867738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad90f3-9a03-4592-8bff-116eeec4cc3c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:47.900422 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:47.947068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b1d41-937b-417c-a1c3-08fe46ba8687 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:47.981643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 414/816] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:34:47.981643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad90f3-9a03-4592-8bff-116eeec4cc3c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:47.988770 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9584a7-f190-4229-bb9e-1612c4d9dac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:47.989383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9584a7-f190-4229-bb9e-1612c4d9dac5 None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:47.990001 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9584a7-f190-4229-bb9e-1612c4d9dac5 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:47.990288 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9584a7-f190-4229-bb9e-1612c4d9dac5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:47.990952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9584a7-f190-4229-bb9e-1612c4d9dac5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:47.992305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9584a7-f190-4229-bb9e-1612c4d9dac5 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:48.077819 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad90f3-9a03-4592-8bff-116eeec4cc3c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:48.102256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:48.104574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:48.115573 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 403/817] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:34:47 2022] POST /identity/v3/endpoints => generated 337 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:34:48.115573 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9584a7-f190-4229-bb9e-1612c4d9dac5 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:48.130983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150d554-bd6f-47e2-b2ac-2956cb98d27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:48.131742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150d554-bd6f-47e2-b2ac-2956cb98d27f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:48.132022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150d554-bd6f-47e2-b2ac-2956cb98d27f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:48.132294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150d554-bd6f-47e2-b2ac-2956cb98d27f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:48.132933 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150d554-bd6f-47e2-b2ac-2956cb98d27f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:48.134474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150d554-bd6f-47e2-b2ac-2956cb98d27f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-e150d554-bd6f-47e2-b2ac-2956cb98d27f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:48.145298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:48.146434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:48.147316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 415/818] 10.222.0.42 () {56 vars in 1129 bytes} [Wed Sep 14 19:34:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:48.148114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150d554-bd6f-47e2-b2ac-2956cb98d27f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:48.248353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9584a7-f190-4229-bb9e-1612c4d9dac5 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:48.264709 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150d554-bd6f-47e2-b2ac-2956cb98d27f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:48.293937 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f6483-4e3d-490f-86eb-208c11313967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:48.294569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f6483-4e3d-490f-86eb-208c11313967 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:48.295187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f6483-4e3d-490f-86eb-208c11313967 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:48.295460 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f6483-4e3d-490f-86eb-208c11313967 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:48.296476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f6483-4e3d-490f-86eb-208c11313967 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:48.296779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f6483-4e3d-490f-86eb-208c11313967 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:48.299222 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f6483-4e3d-490f-86eb-208c11313967 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:48.303289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b168628-b518-446f-b2ab-0e6a08c3247c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:48.304007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168628-b518-446f-b2ab-0e6a08c3247c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:48.304359 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168628-b518-446f-b2ab-0e6a08c3247c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:48.304721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b168628-b518-446f-b2ab-0e6a08c3247c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:48.318138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b168628-b518-446f-b2ab-0e6a08c3247c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:48.319677 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b168628-b518-446f-b2ab-0e6a08c3247c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:48.321589 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 404/819] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Sep 14 19:34:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:48.322347 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f6483-4e3d-490f-86eb-208c11313967 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-3b168628-b518-446f-b2ab-0e6a08c3247c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:48.336059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:48.337086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:48.338023 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:48.338023 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:48.338023 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:48.338023 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 416/820] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:48.338258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b168628-b518-446f-b2ab-0e6a08c3247c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:48.389772 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:48.391506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:48.447798 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f6483-4e3d-490f-86eb-208c11313967 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:48.462892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b168628-b518-446f-b2ab-0e6a08c3247c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:48.488164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8ccca-5143-42d2-9677-243f945990ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:48.488552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8ccca-5143-42d2-9677-243f945990ca None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:48.488831 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8ccca-5143-42d2-9677-243f945990ca None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:48.489105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a8ccca-5143-42d2-9677-243f945990ca None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:48.489728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8ccca-5143-42d2-9677-243f945990ca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:48.490060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8ccca-5143-42d2-9677-243f945990ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:48.491559 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a8ccca-5143-42d2-9677-243f945990ca None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:48.496947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 405/821] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Sep 14 19:34:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:48.498002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8ccca-5143-42d2-9677-243f945990ca None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:48.500397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe01739-60c7-49a0-80d5-669772594f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:48.501048 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe01739-60c7-49a0-80d5-669772594f8e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:48.501476 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe01739-60c7-49a0-80d5-669772594f8e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:48.501924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe01739-60c7-49a0-80d5-669772594f8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:48.513413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe01739-60c7-49a0-80d5-669772594f8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:48.514891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe01739-60c7-49a0-80d5-669772594f8e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-dbe01739-60c7-49a0-80d5-669772594f8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:48.530259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:48.531235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:48.532144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:48.532144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:48.532144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:48.532144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 417/822] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:48.532144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe01739-60c7-49a0-80d5-669772594f8e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:48.607416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a8ccca-5143-42d2-9677-243f945990ca None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:48.628095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:48.631864 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:48.638230 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe01739-60c7-49a0-80d5-669772594f8e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:48.646399 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba2a70-d32f-40c3-80a8-abca6cc10bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:48.647385 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba2a70-d32f-40c3-80a8-abca6cc10bb1 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:48.647799 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba2a70-d32f-40c3-80a8-abca6cc10bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:48.647980 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba2a70-d32f-40c3-80a8-abca6cc10bb1 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:48.648939 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba2a70-d32f-40c3-80a8-abca6cc10bb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:48.649527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba2a70-d32f-40c3-80a8-abca6cc10bb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:48.651080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba2a70-d32f-40c3-80a8-abca6cc10bb1 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:48.671054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 406/823] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:48.671054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba2a70-d32f-40c3-80a8-abca6cc10bb1 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:48.675946 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe86b3d-0f72-4f71-a6fe-1885d9c367f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:48.676562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe86b3d-0f72-4f71-a6fe-1885d9c367f4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:48.682915 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe86b3d-0f72-4f71-a6fe-1885d9c367f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:48.682915 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe86b3d-0f72-4f71-a6fe-1885d9c367f4 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:48.682915 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe86b3d-0f72-4f71-a6fe-1885d9c367f4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:48.682915 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe86b3d-0f72-4f71-a6fe-1885d9c367f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:48.682915 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe86b3d-0f72-4f71-a6fe-1885d9c367f4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:48.689183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 418/824] 10.222.0.42 () {56 vars in 1124 bytes} [Wed Sep 14 19:34:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:48.689183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe86b3d-0f72-4f71-a6fe-1885d9c367f4 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:48.775715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba2a70-d32f-40c3-80a8-abca6cc10bb1 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:48.795709 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe86b3d-0f72-4f71-a6fe-1885d9c367f4 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:48.808516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c89cc-1a66-4072-ab1c-9fd411c0d1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:48.809129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c89cc-1a66-4072-ab1c-9fd411c0d1d3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:48.809413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c89cc-1a66-4072-ab1c-9fd411c0d1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:48.809686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c89cc-1a66-4072-ab1c-9fd411c0d1d3 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:48.810357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c89cc-1a66-4072-ab1c-9fd411c0d1d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:48.810663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c89cc-1a66-4072-ab1c-9fd411c0d1d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:48.812033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c89cc-1a66-4072-ab1c-9fd411c0d1d3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:48.823750 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 407/825] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:48.825077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c89cc-1a66-4072-ab1c-9fd411c0d1d3 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:48.837165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66550a95-feed-4cc7-b3ee-84c66ce6b776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:48.838068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66550a95-feed-4cc7-b3ee-84c66ce6b776 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:48.838586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66550a95-feed-4cc7-b3ee-84c66ce6b776 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:48.839062 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66550a95-feed-4cc7-b3ee-84c66ce6b776 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:48.846619 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66550a95-feed-4cc7-b3ee-84c66ce6b776 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:48.848349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66550a95-feed-4cc7-b3ee-84c66ce6b776 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-66550a95-feed-4cc7-b3ee-84c66ce6b776 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:48.861589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:48.862649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:48.863730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:48.863730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:48.863730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:48.863730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:48.863730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:48.863730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:48.863730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:48.863730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:48.865618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 419/826] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:48.866540 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66550a95-feed-4cc7-b3ee-84c66ce6b776 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:48.879360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:48.889075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:48.937449 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c89cc-1a66-4072-ab1c-9fd411c0d1d3 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:48.974556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2c8b6-0f15-4396-bb68-0033353b8c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:48.975190 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2c8b6-0f15-4396-bb68-0033353b8c70 None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:48.975470 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2c8b6-0f15-4396-bb68-0033353b8c70 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:48.975746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae2c8b6-0f15-4396-bb68-0033353b8c70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:48.976431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae2c8b6-0f15-4396-bb68-0033353b8c70 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:48.977898 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae2c8b6-0f15-4396-bb68-0033353b8c70 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:48.978122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66550a95-feed-4cc7-b3ee-84c66ce6b776 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:49.016447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1753770-351a-45b9-a699-66383902e413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:49.017080 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1753770-351a-45b9-a699-66383902e413 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:49.019025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1753770-351a-45b9-a699-66383902e413 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:49.019025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1753770-351a-45b9-a699-66383902e413 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:49.025775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1753770-351a-45b9-a699-66383902e413 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:49.027302 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1753770-351a-45b9-a699-66383902e413 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-d1753770-351a-45b9-a699-66383902e413 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:49.038402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:49.039446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:49.045365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:49.045365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:49.045365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:49.045365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:49.045365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:49.045365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:49.045365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:49.045365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:49.045365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 420/827] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:49.045365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1753770-351a-45b9-a699-66383902e413 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:49.060556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 408/828] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:34:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:34:49.061521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2c8b6-0f15-4396-bb68-0033353b8c70 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:49.138536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:49.166400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1753770-351a-45b9-a699-66383902e413 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:49.187416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:49.197211 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae2c8b6-0f15-4396-bb68-0033353b8c70 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:49.206258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cf3ae-59d8-4864-9373-34c7b96a7356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:49.206883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cf3ae-59d8-4864-9373-34c7b96a7356 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:49.207207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cf3ae-59d8-4864-9373-34c7b96a7356 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:49.207434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cf3ae-59d8-4864-9373-34c7b96a7356 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:49.208231 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cf3ae-59d8-4864-9373-34c7b96a7356 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:49.208520 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cf3ae-59d8-4864-9373-34c7b96a7356 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:49.209991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cf3ae-59d8-4864-9373-34c7b96a7356 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:49.229111 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 421/829] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:49.229111 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf09327-f85c-4358-b859-374a6ab51068 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:49.229111 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf09327-f85c-4358-b859-374a6ab51068 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:49.229111 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf09327-f85c-4358-b859-374a6ab51068 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:49.229111 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 422/830] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:49.233472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf09327-f85c-4358-b859-374a6ab51068 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:49.243431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b49da1-c664-440c-bed4-30a0ff0d38a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:49.244038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b49da1-c664-440c-bed4-30a0ff0d38a6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:49.244318 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b49da1-c664-440c-bed4-30a0ff0d38a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:49.244589 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b49da1-c664-440c-bed4-30a0ff0d38a6 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:49.245274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b49da1-c664-440c-bed4-30a0ff0d38a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:49.246826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b49da1-c664-440c-bed4-30a0ff0d38a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:49.247629 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b49da1-c664-440c-bed4-30a0ff0d38a6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:49.262767 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 409/831] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:49.262929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414afac-f5ea-4422-8bdd-331a4d5f38a0 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:49.263161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414afac-f5ea-4422-8bdd-331a4d5f38a0 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:49.263433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414afac-f5ea-4422-8bdd-331a4d5f38a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:49.283612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-2414afac-f5ea-4422-8bdd-331a4d5f38a0 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:49.284832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2414afac-f5ea-4422-8bdd-331a4d5f38a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:49.347504 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf09327-f85c-4358-b859-374a6ab51068 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:49.354046 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 410/832] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:34:49.355093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414afac-f5ea-4422-8bdd-331a4d5f38a0 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:49.383194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa18819-c229-444b-8612-55ac14b42b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:49.383799 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa18819-c229-444b-8612-55ac14b42b44 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:49.384078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa18819-c229-444b-8612-55ac14b42b44 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:49.384342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa18819-c229-444b-8612-55ac14b42b44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:49.385044 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa18819-c229-444b-8612-55ac14b42b44 None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=2a4284b59f804dc59674d621d87634bc) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:49.385342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa18819-c229-444b-8612-55ac14b42b44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:49.386800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa18819-c229-444b-8612-55ac14b42b44 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:49.388590 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:49.397507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 423/833] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:49 2022] GET /identity/v3/role_assignments?user.id=2a4284b59f804dc59674d621d87634bc&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:49.461885 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:49.479570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414afac-f5ea-4422-8bdd-331a4d5f38a0 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:49.519362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fa9e8-b12e-409c-8a33-92622ffd52d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:49.520102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fa9e8-b12e-409c-8a33-92622ffd52d0 None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:49.520382 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fa9e8-b12e-409c-8a33-92622ffd52d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:49.520655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fa9e8-b12e-409c-8a33-92622ffd52d0 None admin] PATH_INFO: `/v3/projects/6bfc6c1c443849038af35577daf34088/users/7d5605f645a74b1b842733dda256193c/roles/928c1b8875334543ab431e4f7e2acee5` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:49.538640 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fa9e8-b12e-409c-8a33-92622ffd52d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bfc6c1c443849038af35577daf34088, user_id=7d5605f645a74b1b842733dda256193c, role_id=928c1b8875334543ab431e4f7e2acee5)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:49.540205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fa9e8-b12e-409c-8a33-92622ffd52d0 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:49.656161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 411/834] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:34:49 2022] PUT /identity/v3/projects/6bfc6c1c443849038af35577daf34088/users/7d5605f645a74b1b842733dda256193c/roles/928c1b8875334543ab431e4f7e2acee5 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:34:49.659244 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa18819-c229-444b-8612-55ac14b42b44 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:49.772888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa18819-c229-444b-8612-55ac14b42b44 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:49.812595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820139a8-cce3-47f5-87fb-96564c15c78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:49.813242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820139a8-cce3-47f5-87fb-96564c15c78a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:49.813514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820139a8-cce3-47f5-87fb-96564c15c78a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:49.813771 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820139a8-cce3-47f5-87fb-96564c15c78a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:49.826164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820139a8-cce3-47f5-87fb-96564c15c78a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-820139a8-cce3-47f5-87fb-96564c15c78a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:49.831488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:49.832463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:49.833306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:49.833306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:49.833306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:49.833306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:49.833306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:49.833306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:49.833306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:49.833306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:49.833306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:49.833306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:49.833306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:34:49.833306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:49.833306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 424/835] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:34:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:49.841185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fa9e8-b12e-409c-8a33-92622ffd52d0 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:49.958666 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fa9e8-b12e-409c-8a33-92622ffd52d0 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:49.997439 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba4014-b4d1-435c-ba87-ca36ad25358d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:49.998538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba4014-b4d1-435c-ba87-ca36ad25358d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:49.999032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba4014-b4d1-435c-ba87-ca36ad25358d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:49.999503 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba4014-b4d1-435c-ba87-ca36ad25358d None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:50.000365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba4014-b4d1-435c-ba87-ca36ad25358d None admin] RBAC: Adding query filter params (name=admin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:50.000859 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba4014-b4d1-435c-ba87-ca36ad25358d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:50.002444 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ba4014-b4d1-435c-ba87-ca36ad25358d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:50.004005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:50.012713 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 412/836] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:34:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:50.018979 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820139a8-cce3-47f5-87fb-96564c15c78a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:50.087530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:50.129690 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820139a8-cce3-47f5-87fb-96564c15c78a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:50.163963 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35922f-367b-44a4-bf93-8a9e2d39fed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:50.164594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35922f-367b-44a4-bf93-8a9e2d39fed3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:50.164867 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35922f-367b-44a4-bf93-8a9e2d39fed3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:50.165134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35922f-367b-44a4-bf93-8a9e2d39fed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:50.176425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35922f-367b-44a4-bf93-8a9e2d39fed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:50.177873 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35922f-367b-44a4-bf93-8a9e2d39fed3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-5e35922f-367b-44a4-bf93-8a9e2d39fed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:50.192900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:50.193909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:50.194807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:50.194807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:50.194807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:50.194807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 425/837] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:50.203157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba4014-b4d1-435c-ba87-ca36ad25358d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:50.427724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba4014-b4d1-435c-ba87-ca36ad25358d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:50.463200 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cbbae-2266-4ad7-9dc1-cc469aca2b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:50.463810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cbbae-2266-4ad7-9dc1-cc469aca2b7b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:50.464085 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cbbae-2266-4ad7-9dc1-cc469aca2b7b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:50.464354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cbbae-2266-4ad7-9dc1-cc469aca2b7b None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:50.465517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2cbbae-2266-4ad7-9dc1-cc469aca2b7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:50.465828 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2cbbae-2266-4ad7-9dc1-cc469aca2b7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:50.467216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2cbbae-2266-4ad7-9dc1-cc469aca2b7b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:50.479098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 413/838] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:50.484766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35922f-367b-44a4-bf93-8a9e2d39fed3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:50.589936 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35922f-367b-44a4-bf93-8a9e2d39fed3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:50.600604 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:50.624051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f35aa-ecfe-4d5a-be58-afecc3fd85bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:50.624711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f35aa-ecfe-4d5a-be58-afecc3fd85bc None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:50.624982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f35aa-ecfe-4d5a-be58-afecc3fd85bc None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:50.625249 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f35aa-ecfe-4d5a-be58-afecc3fd85bc None admin] PATH_INFO: `/v3/users/swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:50.633000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f35aa-ecfe-4d5a-be58-afecc3fd85bc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:50.634605 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f35aa-ecfe-4d5a-be58-afecc3fd85bc None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-6e0f35aa-ecfe-4d5a-be58-afecc3fd85bc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:50.646945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:50.648015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:50.648972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:50.648972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:50.648972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:50.648972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:50.648972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:50.648972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:50.648972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:50.648972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:50.648972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 19:34:50.648972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:50.648972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 426/839] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:34:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:50.659365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cbbae-2266-4ad7-9dc1-cc469aca2b7b None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:50.732155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:50.779351 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cbbae-2266-4ad7-9dc1-cc469aca2b7b None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:50.813242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966995a-326a-448f-ab88-6b9e1f69cd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:50.813869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966995a-326a-448f-ab88-6b9e1f69cd14 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:50.814151 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966995a-326a-448f-ab88-6b9e1f69cd14 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:50.814411 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e966995a-326a-448f-ab88-6b9e1f69cd14 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:50.815132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966995a-326a-448f-ab88-6b9e1f69cd14 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:50.815432 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966995a-326a-448f-ab88-6b9e1f69cd14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:50.816844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e966995a-326a-448f-ab88-6b9e1f69cd14 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:50.834840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 414/840] 10.222.0.42 () {56 vars in 1135 bytes} [Wed Sep 14 19:34:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:50.841114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f35aa-ecfe-4d5a-be58-afecc3fd85bc None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:50.950745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f35aa-ecfe-4d5a-be58-afecc3fd85bc None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:50.986021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a868a85-b6ad-4b8d-b718-5679aa69e87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:50.986607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a868a85-b6ad-4b8d-b718-5679aa69e87b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:50.986878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a868a85-b6ad-4b8d-b718-5679aa69e87b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:50.987144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a868a85-b6ad-4b8d-b718-5679aa69e87b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:51.026320 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:51.054469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a868a85-b6ad-4b8d-b718-5679aa69e87b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:51.055911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a868a85-b6ad-4b8d-b718-5679aa69e87b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:51.158808 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 427/841] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:50 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:34:51.164736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966995a-326a-448f-ab88-6b9e1f69cd14 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:51.262871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0c4ff-cc57-44aa-87dd-3b3c8c2bc4f1 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:51.263045 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0c4ff-cc57-44aa-87dd-3b3c8c2bc4f1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:51.263327 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb0c4ff-cc57-44aa-87dd-3b3c8c2bc4f1 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:51.265998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 428/842] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:34:51.269869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1be02-c3d3-4e60-ad8b-ff5bc61a8bae None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:51.270149 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1be02-c3d3-4e60-ad8b-ff5bc61a8bae None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:51.270417 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1be02-c3d3-4e60-ad8b-ff5bc61a8bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:51.274052 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e966995a-326a-448f-ab88-6b9e1f69cd14 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:51.288987 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f8b1be02-c3d3-4e60-ad8b-ff5bc61a8bae None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:51.290194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f8b1be02-c3d3-4e60-ad8b-ff5bc61a8bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:51.295304 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:51.314499 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06aaa5-d594-48df-889a-31abb5e7bc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:51.315413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06aaa5-d594-48df-889a-31abb5e7bc8c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:51.315690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06aaa5-d594-48df-889a-31abb5e7bc8c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:51.316249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed06aaa5-d594-48df-889a-31abb5e7bc8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:51.330772 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06aaa5-d594-48df-889a-31abb5e7bc8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:51.332283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed06aaa5-d594-48df-889a-31abb5e7bc8c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-ed06aaa5-d594-48df-889a-31abb5e7bc8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:51.351520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:51.352684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:51.353655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:51.353655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:51.353655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:51.354363 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 415/843] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:51.359994 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06aaa5-d594-48df-889a-31abb5e7bc8c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:51.368702 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:51.379026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 429/844] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:34:51.468545 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed06aaa5-d594-48df-889a-31abb5e7bc8c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:51.503417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed467e3-077f-43d1-b884-7483dfa25943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:51.504084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed467e3-077f-43d1-b884-7483dfa25943 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:51.504360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed467e3-077f-43d1-b884-7483dfa25943 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:51.504690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed467e3-077f-43d1-b884-7483dfa25943 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:51.505328 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed467e3-077f-43d1-b884-7483dfa25943 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:51.505627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed467e3-077f-43d1-b884-7483dfa25943 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:51.507127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed467e3-077f-43d1-b884-7483dfa25943 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:51.519286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 416/845] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:51.526917 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1be02-c3d3-4e60-ad8b-ff5bc61a8bae None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:51.637169 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1be02-c3d3-4e60-ad8b-ff5bc61a8bae None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:51.666986 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed467e3-077f-43d1-b884-7483dfa25943 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:51.673300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf13652-c168-4243-b9f8-d871b5cae2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:51.673986 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf13652-c168-4243-b9f8-d871b5cae2b3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:51.674267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf13652-c168-4243-b9f8-d871b5cae2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:51.674537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf13652-c168-4243-b9f8-d871b5cae2b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:51.682544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf13652-c168-4243-b9f8-d871b5cae2b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:51.684548 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf13652-c168-4243-b9f8-d871b5cae2b3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:51.699912 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-0bf13652-c168-4243-b9f8-d871b5cae2b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:51.700071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:51.700938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:51.701902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:51.701902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:51.701902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:51.701902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:51.701902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:51.701902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:51.701902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:51.701902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:51.701902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 430/846] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:51.708760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf13652-c168-4243-b9f8-d871b5cae2b3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:51.772836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:51.786236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed467e3-077f-43d1-b884-7483dfa25943 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:51.815680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf13652-c168-4243-b9f8-d871b5cae2b3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:51.821942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905c0d8-e02f-4754-8112-55059b1479df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:51.822572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905c0d8-e02f-4754-8112-55059b1479df None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:51.822852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905c0d8-e02f-4754-8112-55059b1479df None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:51.823113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905c0d8-e02f-4754-8112-55059b1479df None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:51.823687 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905c0d8-e02f-4754-8112-55059b1479df None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:51.825549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905c0d8-e02f-4754-8112-55059b1479df None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-2905c0d8-e02f-4754-8112-55059b1479df None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:51.834254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:51.835461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:51.836314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 417/847] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:34:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:51.837108 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47b406-9d85-49d1-ad65-9c450688df19 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:51.837108 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47b406-9d85-49d1-ad65-9c450688df19 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:51.837782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47b406-9d85-49d1-ad65-9c450688df19 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:51.841845 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 418/848] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:51.842314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47b406-9d85-49d1-ad65-9c450688df19 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:51.862361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d74549-0654-4552-a0d4-c60a8d1b1c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:51.862973 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d74549-0654-4552-a0d4-c60a8d1b1c49 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:51.863249 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d74549-0654-4552-a0d4-c60a8d1b1c49 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:51.863516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d74549-0654-4552-a0d4-c60a8d1b1c49 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:51.864191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d74549-0654-4552-a0d4-c60a8d1b1c49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:51.864495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d74549-0654-4552-a0d4-c60a8d1b1c49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:51.866402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d74549-0654-4552-a0d4-c60a8d1b1c49 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:51.883240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 431/849] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:51.887793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d0dd8-b751-4c3b-a11d-597d2c478080 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:51.888919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d0dd8-b751-4c3b-a11d-597d2c478080 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:51.889206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40d0dd8-b751-4c3b-a11d-597d2c478080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:51.910166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f40d0dd8-b751-4c3b-a11d-597d2c478080 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:51.911450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f40d0dd8-b751-4c3b-a11d-597d2c478080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:51.950187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:51.961036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47b406-9d85-49d1-ad65-9c450688df19 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:51.993821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 432/850] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:34:51.993821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d0dd8-b751-4c3b-a11d-597d2c478080 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:52.012180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a4def-722d-4c93-b9da-084989cb0836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:52.013051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a4def-722d-4c93-b9da-084989cb0836 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:52.013343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a4def-722d-4c93-b9da-084989cb0836 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:52.013615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a4def-722d-4c93-b9da-084989cb0836 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:52.014401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a4def-722d-4c93-b9da-084989cb0836 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:52.014725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a4def-722d-4c93-b9da-084989cb0836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:52.016553 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a4def-722d-4c93-b9da-084989cb0836 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:52.021204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 419/851] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:34:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:52.022379 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67cabe-3539-4354-a565-c9a530e350e2 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:52.022657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67cabe-3539-4354-a565-c9a530e350e2 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:52.022978 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67cabe-3539-4354-a565-c9a530e350e2 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:52.024109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 420/852] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:52.028657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67cabe-3539-4354-a565-c9a530e350e2 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:52.041549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:52.094907 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40d0dd8-b751-4c3b-a11d-597d2c478080 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:52.129914 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0fe12-1943-4d6b-83a6-2041c081c529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:52.130514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0fe12-1943-4d6b-83a6-2041c081c529 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:52.130783 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0fe12-1943-4d6b-83a6-2041c081c529 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:52.131046 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0fe12-1943-4d6b-83a6-2041c081c529 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:52.131746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0fe12-1943-4d6b-83a6-2041c081c529 None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=e3d2f1ab1bfd435183e33168400c0b17) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:52.132044 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0fe12-1943-4d6b-83a6-2041c081c529 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:52.133453 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0fe12-1943-4d6b-83a6-2041c081c529 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:52.134983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67cabe-3539-4354-a565-c9a530e350e2 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:52.139470 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 433/853] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:51 2022] GET /identity/v3/role_assignments?user.id=e3d2f1ab1bfd435183e33168400c0b17&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:52.140766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bb9c3-1ced-41ab-a915-445b5512ecfa None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:52.141174 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bb9c3-1ced-41ab-a915-445b5512ecfa None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:52.141569 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bb9c3-1ced-41ab-a915-445b5512ecfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:52.164069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-dd6bb9c3-1ced-41ab-a915-445b5512ecfa None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:52.165293 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-dd6bb9c3-1ced-41ab-a915-445b5512ecfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:52.198575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70b068-1c0d-4324-955a-145d1fbc7597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:52.200283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70b068-1c0d-4324-955a-145d1fbc7597 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:52.200841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70b068-1c0d-4324-955a-145d1fbc7597 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:52.201383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70b068-1c0d-4324-955a-145d1fbc7597 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:52.202311 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70b068-1c0d-4324-955a-145d1fbc7597 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:52.202861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70b068-1c0d-4324-955a-145d1fbc7597 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:52.204502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b70b068-1c0d-4324-955a-145d1fbc7597 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:52.211747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 421/854] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:34:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:52.248259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 434/855] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:34:52.265098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70b068-1c0d-4324-955a-145d1fbc7597 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:52.286702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:52.388321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b70b068-1c0d-4324-955a-145d1fbc7597 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:52.425232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2380d43-8909-491a-b334-e0e7a7e3aa5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:52.425889 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2380d43-8909-491a-b334-e0e7a7e3aa5d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:52.426164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2380d43-8909-491a-b334-e0e7a7e3aa5d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:52.426431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2380d43-8909-491a-b334-e0e7a7e3aa5d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:52.427004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2380d43-8909-491a-b334-e0e7a7e3aa5d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:52.428341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2380d43-8909-491a-b334-e0e7a7e3aa5d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-d2380d43-8909-491a-b334-e0e7a7e3aa5d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:52.437317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:52.438266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:52.439755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 422/856] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Sep 14 19:34:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:52.444600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bb9c3-1ced-41ab-a915-445b5512ecfa None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:52.495702 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:52.537668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2380d43-8909-491a-b334-e0e7a7e3aa5d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:52.560725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bb9c3-1ced-41ab-a915-445b5512ecfa None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:52.608889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0ba09-7f53-4f72-9de3-808066d85bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:52.609510 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0ba09-7f53-4f72-9de3-808066d85bc6 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:52.609787 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0ba09-7f53-4f72-9de3-808066d85bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:52.610080 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0ba09-7f53-4f72-9de3-808066d85bc6 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:52.611106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0ba09-7f53-4f72-9de3-808066d85bc6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:52.611408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0ba09-7f53-4f72-9de3-808066d85bc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:52.613092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb0ba09-7f53-4f72-9de3-808066d85bc6 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:52.620366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 435/857] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 19:34:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:52.624704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0ba09-7f53-4f72-9de3-808066d85bc6 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:52.634957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:52.655278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2380d43-8909-491a-b334-e0e7a7e3aa5d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:52.700541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5521d1b-a2be-459e-b239-39de570ef1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:52.701186 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5521d1b-a2be-459e-b239-39de570ef1e5 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:52.701685 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5521d1b-a2be-459e-b239-39de570ef1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:52.702058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5521d1b-a2be-459e-b239-39de570ef1e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:52.716190 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5521d1b-a2be-459e-b239-39de570ef1e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-d5521d1b-a2be-459e-b239-39de570ef1e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:52.722759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:52.723966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:52.724858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:52.724858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:52.724858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:52.724858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:52.724858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:52.724858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:52.724858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:52.724858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:52.724858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:52.724858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:52.724858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:34:52.724858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:52.724858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 423/858] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:34:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:52.731098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5521d1b-a2be-459e-b239-39de570ef1e5 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:52.747960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0ba09-7f53-4f72-9de3-808066d85bc6 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:52.762329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:52.795043 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cabf28-8ea1-4082-b063-313ab35a8584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:52.795597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cabf28-8ea1-4082-b063-313ab35a8584 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:52.795873 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cabf28-8ea1-4082-b063-313ab35a8584 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:52.796147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cabf28-8ea1-4082-b063-313ab35a8584 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:52.796768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cabf28-8ea1-4082-b063-313ab35a8584 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:52.797070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cabf28-8ea1-4082-b063-313ab35a8584 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:52.798582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cabf28-8ea1-4082-b063-313ab35a8584 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:52.809350 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 436/859] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 19:34:52 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:52.859483 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5521d1b-a2be-459e-b239-39de570ef1e5 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:52.898130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c05788-2d7c-49f5-8ced-5ec6a73bf861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:52.899131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c05788-2d7c-49f5-8ced-5ec6a73bf861 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:52.899611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c05788-2d7c-49f5-8ced-5ec6a73bf861 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:52.900096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c05788-2d7c-49f5-8ced-5ec6a73bf861 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:52.900977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c05788-2d7c-49f5-8ced-5ec6a73bf861 None admin] RBAC: Adding query filter params (name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:52.901487 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c05788-2d7c-49f5-8ced-5ec6a73bf861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:52.903803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c05788-2d7c-49f5-8ced-5ec6a73bf861 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:52.914060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 424/860] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:34:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:52.921940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cabf28-8ea1-4082-b063-313ab35a8584 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:53.035424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cabf28-8ea1-4082-b063-313ab35a8584 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:53.072031 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e5686-19cd-45a4-aab1-5249b7b61721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:53.072588 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e5686-19cd-45a4-aab1-5249b7b61721 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:53.072864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e5686-19cd-45a4-aab1-5249b7b61721 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:53.073132 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3e5686-19cd-45a4-aab1-5249b7b61721 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:53.083860 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e5686-19cd-45a4-aab1-5249b7b61721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:53.085291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3e5686-19cd-45a4-aab1-5249b7b61721 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-1a3e5686-19cd-45a4-aab1-5249b7b61721 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:53.100033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:53.100977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:53.102269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:53.102269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:53.102269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:53.102269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 437/861] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:53.107697 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c05788-2d7c-49f5-8ced-5ec6a73bf861 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:53.112129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:53.219185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c05788-2d7c-49f5-8ced-5ec6a73bf861 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:53.254090 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531c565-aff9-4f8d-ac6d-acb30f69b457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:53.254932 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c565-aff9-4f8d-ac6d-acb30f69b457 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:53.255355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c565-aff9-4f8d-ac6d-acb30f69b457 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:53.255814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b531c565-aff9-4f8d-ac6d-acb30f69b457 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:53.256582 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531c565-aff9-4f8d-ac6d-acb30f69b457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:53.257123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531c565-aff9-4f8d-ac6d-acb30f69b457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:53.258746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b531c565-aff9-4f8d-ac6d-acb30f69b457 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:53.271472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 425/862] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:53.277969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e5686-19cd-45a4-aab1-5249b7b61721 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:53.303209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:53.380458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3e5686-19cd-45a4-aab1-5249b7b61721 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:53.415684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682a675-00db-4fad-b876-2ad1172cd7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:53.416659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682a675-00db-4fad-b876-2ad1172cd7bb None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:53.417149 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682a675-00db-4fad-b876-2ad1172cd7bb None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:53.417658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0682a675-00db-4fad-b876-2ad1172cd7bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:53.425546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682a675-00db-4fad-b876-2ad1172cd7bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:53.427306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0682a675-00db-4fad-b876-2ad1172cd7bb None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-0682a675-00db-4fad-b876-2ad1172cd7bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:53.439355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:53.440356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:53.441352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:53.441352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:53.441352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:53.441352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:53.441352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:53.441352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:53.441352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:53.441352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:53.441352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 19:34:53.441352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:53.444322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 438/863] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 19:34:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:53.450843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531c565-aff9-4f8d-ac6d-acb30f69b457 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:53.515272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:53.553836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b531c565-aff9-4f8d-ac6d-acb30f69b457 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:53.591496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02346fcf-3f59-4c6c-bcea-fb98f6f4b849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:53.592106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02346fcf-3f59-4c6c-bcea-fb98f6f4b849 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:53.592382 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02346fcf-3f59-4c6c-bcea-fb98f6f4b849 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:53.592649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02346fcf-3f59-4c6c-bcea-fb98f6f4b849 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:53.593370 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02346fcf-3f59-4c6c-bcea-fb98f6f4b849 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:53.593671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02346fcf-3f59-4c6c-bcea-fb98f6f4b849 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:53.595112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02346fcf-3f59-4c6c-bcea-fb98f6f4b849 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:53.613117 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 426/864] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Sep 14 19:34:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:53.618779 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682a675-00db-4fad-b876-2ad1172cd7bb None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:53.707416 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:53.725723 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0682a675-00db-4fad-b876-2ad1172cd7bb None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:53.761100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88019881-ce34-40b5-ab95-7c4398c6cef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:53.762046 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88019881-ce34-40b5-ab95-7c4398c6cef4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:53.762546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88019881-ce34-40b5-ab95-7c4398c6cef4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:53.763022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88019881-ce34-40b5-ab95-7c4398c6cef4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:53.832883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88019881-ce34-40b5-ab95-7c4398c6cef4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:53.834846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88019881-ce34-40b5-ab95-7c4398c6cef4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:53.930287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:53.947460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 439/865] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:53 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:34:53.953561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02346fcf-3f59-4c6c-bcea-fb98f6f4b849 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:54.061358 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02346fcf-3f59-4c6c-bcea-fb98f6f4b849 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:54.085336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:54.096984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c41081-e278-4b84-bfd3-47ff6211df3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:54.100222 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c41081-e278-4b84-bfd3-47ff6211df3f None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:54.100222 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c41081-e278-4b84-bfd3-47ff6211df3f None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:54.100222 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c41081-e278-4b84-bfd3-47ff6211df3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:54.108890 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c41081-e278-4b84-bfd3-47ff6211df3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:54.110370 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c41081-e278-4b84-bfd3-47ff6211df3f None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-d3c41081-e278-4b84-bfd3-47ff6211df3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:54.125008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:54.126002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:54.126903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:54.126903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:54.126903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:54.126903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 427/866] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:54.131649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88019881-ce34-40b5-ab95-7c4398c6cef4 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:54.237555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88019881-ce34-40b5-ab95-7c4398c6cef4 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:54.273466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494d185-6ebc-4962-a834-fcf72d48f3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:54.274102 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494d185-6ebc-4962-a834-fcf72d48f3a3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:54.274378 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494d185-6ebc-4962-a834-fcf72d48f3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:54.274648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494d185-6ebc-4962-a834-fcf72d48f3a3 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:54.275276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494d185-6ebc-4962-a834-fcf72d48f3a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:54.275632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494d185-6ebc-4962-a834-fcf72d48f3a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:54.277524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494d185-6ebc-4962-a834-fcf72d48f3a3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:54.290712 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 440/867] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:54.296527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c41081-e278-4b84-bfd3-47ff6211df3f None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:54.407785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c41081-e278-4b84-bfd3-47ff6211df3f None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:54.410089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40ed9e-9b0c-40ee-a7c1-a1798fd64bbf None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:54.410370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40ed9e-9b0c-40ee-a7c1-a1798fd64bbf None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:54.410643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40ed9e-9b0c-40ee-a7c1-a1798fd64bbf None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:54.412306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 441/868] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:54.418137 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff48f4-2761-43dc-b8f7-86911d7595d8 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:54.418468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff48f4-2761-43dc-b8f7-86911d7595d8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:54.418676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff48f4-2761-43dc-b8f7-86911d7595d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:54.438042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e3ff48f4-2761-43dc-b8f7-86911d7595d8 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:54.439333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e3ff48f4-2761-43dc-b8f7-86911d7595d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:54.449903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c5077-512c-4259-a691-40a2c90e9223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:54.450626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c5077-512c-4259-a691-40a2c90e9223 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:54.450976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c5077-512c-4259-a691-40a2c90e9223 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:54.451322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c5077-512c-4259-a691-40a2c90e9223 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:54.458243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c5077-512c-4259-a691-40a2c90e9223 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:54.459763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c5077-512c-4259-a691-40a2c90e9223 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-8c9c5077-512c-4259-a691-40a2c90e9223 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:54.471435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:54.472325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:54.473231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:54.473231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:54.473231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:54.473231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:54.473231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:54.473231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:54.473231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:54.473231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:54.473231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 428/869] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:54.477949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c5077-512c-4259-a691-40a2c90e9223 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:54.488060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:54.518672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:54.525905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 442/870] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:34:54.535324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539d3fa-4e0c-49c7-a3e8-2176b562bef0 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:54.535656 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539d3fa-4e0c-49c7-a3e8-2176b562bef0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:54.535966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539d3fa-4e0c-49c7-a3e8-2176b562bef0 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:54.537419 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 443/871] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:34:54.542566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6199c16-2832-4c5b-b750-88502ce2abd9 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:54.543718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6199c16-2832-4c5b-b750-88502ce2abd9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:54.544446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6199c16-2832-4c5b-b750-88502ce2abd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:54.564895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e6199c16-2832-4c5b-b750-88502ce2abd9 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:54.566488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e6199c16-2832-4c5b-b750-88502ce2abd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:54.610515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c5077-512c-4259-a691-40a2c90e9223 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:54.649842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 444/872] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:34:54.650384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6da955-a0ba-426e-a536-d4f9a8aa0228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:54.650384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6da955-a0ba-426e-a536-d4f9a8aa0228 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:54.650384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6da955-a0ba-426e-a536-d4f9a8aa0228 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:54.650384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6da955-a0ba-426e-a536-d4f9a8aa0228 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:54.650670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6da955-a0ba-426e-a536-d4f9a8aa0228 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:54.650670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6da955-a0ba-426e-a536-d4f9a8aa0228 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:54.650670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6da955-a0ba-426e-a536-d4f9a8aa0228 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:54.662473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 429/873] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:54.670861 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6199c16-2832-4c5b-b750-88502ce2abd9 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:54.775953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6199c16-2832-4c5b-b750-88502ce2abd9 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:54.778675 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:54.792458 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6da955-a0ba-426e-a536-d4f9a8aa0228 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:54.820184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104eccff-320a-4948-b65f-f7dafc2834a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:54.821080 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104eccff-320a-4948-b65f-f7dafc2834a9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:54.821953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104eccff-320a-4948-b65f-f7dafc2834a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:54.822258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104eccff-320a-4948-b65f-f7dafc2834a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:54.823098 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104eccff-320a-4948-b65f-f7dafc2834a9 None admin] RBAC: Adding query filter params (role.id=928c1b8875334543ab431e4f7e2acee5, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=7d5605f645a74b1b842733dda256193c) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:54.823405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104eccff-320a-4948-b65f-f7dafc2834a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:54.826059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104eccff-320a-4948-b65f-f7dafc2834a9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:54.834432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 445/874] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:34:54 2022] GET /identity/v3/role_assignments?user.id=7d5605f645a74b1b842733dda256193c&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=928c1b8875334543ab431e4f7e2acee5 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:54.913351 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:54.921889 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6da955-a0ba-426e-a536-d4f9a8aa0228 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:54.960541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc14916-6c24-4818-a399-23ca5ffd85e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:54.961210 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104eccff-320a-4948-b65f-f7dafc2834a9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:54.961304 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc14916-6c24-4818-a399-23ca5ffd85e7 None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:54.961552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc14916-6c24-4818-a399-23ca5ffd85e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:54.961843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc14916-6c24-4818-a399-23ca5ffd85e7 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:54.962521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc14916-6c24-4818-a399-23ca5ffd85e7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:54.964297 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc14916-6c24-4818-a399-23ca5ffd85e7 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:55.055395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 430/875] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 19:34:54 2022] POST /identity/v3/services => generated 251 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:34:55.090212 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104eccff-320a-4948-b65f-f7dafc2834a9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:55.138370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19819361-c1fe-4dc0-ad8f-f6ef63c30253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:55.139109 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19819361-c1fe-4dc0-ad8f-f6ef63c30253 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:55.139354 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19819361-c1fe-4dc0-ad8f-f6ef63c30253 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:55.139618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19819361-c1fe-4dc0-ad8f-f6ef63c30253 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:55.140200 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19819361-c1fe-4dc0-ad8f-f6ef63c30253 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:55.141594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19819361-c1fe-4dc0-ad8f-f6ef63c30253 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-19819361-c1fe-4dc0-ad8f-f6ef63c30253 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:55.153289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:55.154385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:55.155236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 446/876] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 19:34:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:55.160909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc14916-6c24-4818-a399-23ca5ffd85e7 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:55.175772 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb411f0e-012e-4ea8-9096-a6f96f02d67b None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:55.176338 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb411f0e-012e-4ea8-9096-a6f96f02d67b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:55.176805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb411f0e-012e-4ea8-9096-a6f96f02d67b None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:55.178647 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 447/877] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:55.184905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb799dc-c4bd-4eb9-8e31-109f247cf06b None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:55.185508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb799dc-c4bd-4eb9-8e31-109f247cf06b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:55.186028 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb799dc-c4bd-4eb9-8e31-109f247cf06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:55.206681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8eb799dc-c4bd-4eb9-8e31-109f247cf06b None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:55.208261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8eb799dc-c4bd-4eb9-8e31-109f247cf06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:55.236148 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:55.295751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc14916-6c24-4818-a399-23ca5ffd85e7 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:55.296857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 448/878] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:34:55.332367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b11d72-68c0-44a3-9af4-fa4891c2bd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:55.333412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b11d72-68c0-44a3-9af4-fa4891c2bd99 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:55.333941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b11d72-68c0-44a3-9af4-fa4891c2bd99 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:55.334456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b11d72-68c0-44a3-9af4-fa4891c2bd99 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:55.335274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b11d72-68c0-44a3-9af4-fa4891c2bd99 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:55.335780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b11d72-68c0-44a3-9af4-fa4891c2bd99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:55.337456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b11d72-68c0-44a3-9af4-fa4891c2bd99 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:55.342722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 431/879] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 19:34:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:55.350720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb799dc-c4bd-4eb9-8e31-109f247cf06b None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:55.455452 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb799dc-c4bd-4eb9-8e31-109f247cf06b None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:55.489763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf901c10-53b3-4fb9-a717-b3a4fe752932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:55.490390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf901c10-53b3-4fb9-a717-b3a4fe752932 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:55.490662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf901c10-53b3-4fb9-a717-b3a4fe752932 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:55.490923 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf901c10-53b3-4fb9-a717-b3a4fe752932 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:55.491536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf901c10-53b3-4fb9-a717-b3a4fe752932 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:55.491831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf901c10-53b3-4fb9-a717-b3a4fe752932 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:55.493179 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf901c10-53b3-4fb9-a717-b3a4fe752932 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:55.498748 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 449/880] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 19:34:55 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:55.594993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b11d72-68c0-44a3-9af4-fa4891c2bd99 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:55.616430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:55.706823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b11d72-68c0-44a3-9af4-fa4891c2bd99 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:55.742327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d96b79b-d5ea-4324-b2ad-a093668c13df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:55.742928 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96b79b-d5ea-4324-b2ad-a093668c13df None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:55.743206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96b79b-d5ea-4324-b2ad-a093668c13df None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:55.743475 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d96b79b-d5ea-4324-b2ad-a093668c13df None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:55.744223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96b79b-d5ea-4324-b2ad-a093668c13df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:55.745566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d96b79b-d5ea-4324-b2ad-a093668c13df None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:55.838969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 432/881] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 19:34:55 2022] POST /identity/v3/services => generated 269 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:34:57.157679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308f0bd-c256-424f-b3b2-0ab5ecf98e3e None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:57.158192 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308f0bd-c256-424f-b3b2-0ab5ecf98e3e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:57.158192 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308f0bd-c256-424f-b3b2-0ab5ecf98e3e None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:57.159726 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 450/882] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:57.165222 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc0361-9a7d-45a9-ab6c-ea2b1ed104e3 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:57.165628 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc0361-9a7d-45a9-ab6c-ea2b1ed104e3 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:57.165851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc0361-9a7d-45a9-ab6c-ea2b1ed104e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:57.186327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-f0fc0361-9a7d-45a9-ab6c-ea2b1ed104e3 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:57.187557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-f0fc0361-9a7d-45a9-ab6c-ea2b1ed104e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:57.268127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 433/883] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:34:57.296432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ca7c7-fa19-4ce6-ba05-067e74fed263 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:57.296720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ca7c7-fa19-4ce6-ba05-067e74fed263 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:57.296982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89ca7c7-fa19-4ce6-ba05-067e74fed263 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:57.298591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 451/884] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:57.303663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd111-a68f-4a8b-aa45-cecfa272fb2e None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:57.303945 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd111-a68f-4a8b-aa45-cecfa272fb2e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:57.304210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fdd111-a68f-4a8b-aa45-cecfa272fb2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:57.312043 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:57.323659 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-98fdd111-a68f-4a8b-aa45-cecfa272fb2e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:57.329387 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-98fdd111-a68f-4a8b-aa45-cecfa272fb2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:57.393862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 434/885] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:34:57.541480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ca7c7-fa19-4ce6-ba05-067e74fed263 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:57.566565 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:57.650131 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89ca7c7-fa19-4ce6-ba05-067e74fed263 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:57.664423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdd111-a68f-4a8b-aa45-cecfa272fb2e None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:57.690546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2c12f-0be9-46e4-aa9e-ba46f6d1de54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:57.691235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2c12f-0be9-46e4-aa9e-ba46f6d1de54 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:57.692075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2c12f-0be9-46e4-aa9e-ba46f6d1de54 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:57.692602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2c12f-0be9-46e4-aa9e-ba46f6d1de54 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:57.706064 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2c12f-0be9-46e4-aa9e-ba46f6d1de54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-a7b2c12f-0be9-46e4-aa9e-ba46f6d1de54 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:34:57.711984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:57.712924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:57.713751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:57.713751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:57.713751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:57.713751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:57.713751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:34:57.713751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:34:57.713751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:34:57.713751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:34:57.713751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:34:57.713751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:34:57.713751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 19:34:57.713751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:57.714540 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 452/886] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Sep 14 19:34:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:57.722879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2c12f-0be9-46e4-aa9e-ba46f6d1de54 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:57.786467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fdd111-a68f-4a8b-aa45-cecfa272fb2e None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:57.814940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:57.824319 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad6ee9-a222-40b0-b719-c48451824340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:57.826696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad6ee9-a222-40b0-b719-c48451824340 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:57.826696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad6ee9-a222-40b0-b719-c48451824340 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:57.826696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad6ee9-a222-40b0-b719-c48451824340 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:57.826696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad6ee9-a222-40b0-b719-c48451824340 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:57.828075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad6ee9-a222-40b0-b719-c48451824340 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:57.830193 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:57.833948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2c12f-0be9-46e4-aa9e-ba46f6d1de54 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-85ad6ee9-a222-40b0-b719-c48451824340 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:57.838217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:57.839454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:57.840549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:57.840549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:57.840549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:57.840549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:57.840549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:57.840549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:57.840549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:57.840549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:57.840549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:57.840549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:57.840549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:34:57.840549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:57.840549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 19:34:57.840549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:57.843021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 435/887] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:34:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:57.844327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92667ea8-a541-42d3-9af5-9a09bcab005c None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:57.844604 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92667ea8-a541-42d3-9af5-9a09bcab005c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:57.844872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92667ea8-a541-42d3-9af5-9a09bcab005c None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:57.850423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 436/888] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:57.850423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92667ea8-a541-42d3-9af5-9a09bcab005c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:57.877353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f476c-86cf-4b15-85bb-06d62e8c34ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:57.878144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f476c-86cf-4b15-85bb-06d62e8c34ea None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:57.878709 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f476c-86cf-4b15-85bb-06d62e8c34ea None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:57.878988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f476c-86cf-4b15-85bb-06d62e8c34ea None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:57.879638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f476c-86cf-4b15-85bb-06d62e8c34ea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:57.879942 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f476c-86cf-4b15-85bb-06d62e8c34ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:57.881720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f476c-86cf-4b15-85bb-06d62e8c34ea None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:57.890993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 453/889] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Sep 14 19:34:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:57.895246 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e324c-6c9b-4666-bfa2-c3cc47d92ae2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:57.895584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e324c-6c9b-4666-bfa2-c3cc47d92ae2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:57.896236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e324c-6c9b-4666-bfa2-c3cc47d92ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:57.916147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-252e324c-6c9b-4666-bfa2-c3cc47d92ae2 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:57.917263 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-252e324c-6c9b-4666-bfa2-c3cc47d92ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:57.967784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92667ea8-a541-42d3-9af5-9a09bcab005c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:57.993865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 454/890] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:34:57.993865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e324c-6c9b-4666-bfa2-c3cc47d92ae2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:58.009736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f514e-2a6b-4c04-a1df-28f3a7a35796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:58.010442 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f514e-2a6b-4c04-a1df-28f3a7a35796 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:58.010794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f514e-2a6b-4c04-a1df-28f3a7a35796 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:58.011151 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f514e-2a6b-4c04-a1df-28f3a7a35796 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:58.011985 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f514e-2a6b-4c04-a1df-28f3a7a35796 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:58.012356 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f514e-2a6b-4c04-a1df-28f3a7a35796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:58.013950 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f514e-2a6b-4c04-a1df-28f3a7a35796 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:58.023804 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 437/891] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:34:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:58.024716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f514e-2a6b-4c04-a1df-28f3a7a35796 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:58.088214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:58.090518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:58.112612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e324c-6c9b-4666-bfa2-c3cc47d92ae2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:58.128150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f514e-2a6b-4c04-a1df-28f3a7a35796 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:58.148544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0af49a-a921-4ee9-896a-5970ca4ecee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:58.149199 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0af49a-a921-4ee9-896a-5970ca4ecee9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:58.149485 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0af49a-a921-4ee9-896a-5970ca4ecee9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:58.149748 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0af49a-a921-4ee9-896a-5970ca4ecee9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:58.160136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0af49a-a921-4ee9-896a-5970ca4ecee9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:58.161571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0af49a-a921-4ee9-896a-5970ca4ecee9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:58.162370 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8484c-2f13-452f-ab7a-084faf13c480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:58.162974 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8484c-2f13-452f-ab7a-084faf13c480 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:58.163363 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8484c-2f13-452f-ab7a-084faf13c480 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:58.163633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8484c-2f13-452f-ab7a-084faf13c480 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:58.164254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8484c-2f13-452f-ab7a-084faf13c480 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:58.164552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8484c-2f13-452f-ab7a-084faf13c480 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:58.165906 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8484c-2f13-452f-ab7a-084faf13c480 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:58.175295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 438/892] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:34:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:58.175295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59b684-a456-4ccd-9e96-80f5a7443c11 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:58.175295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59b684-a456-4ccd-9e96-80f5a7443c11 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:58.175295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e59b684-a456-4ccd-9e96-80f5a7443c11 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:58.175295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 439/893] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:34:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:34:58.178245 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59b684-a456-4ccd-9e96-80f5a7443c11 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-2f0af49a-a921-4ee9-896a-5970ca4ecee9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:58.181223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:58.182157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:58.183619 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:58.183619 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:58.183619 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:58.183619 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 455/894] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:58.184253 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc769e-a48b-438a-9e72-77897c5483cb None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:58.184253 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc769e-a48b-438a-9e72-77897c5483cb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:58.184531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc769e-a48b-438a-9e72-77897c5483cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:58.203968 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7bfc769e-a48b-438a-9e72-77897c5483cb None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:34:58.205154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7bfc769e-a48b-438a-9e72-77897c5483cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:34:58.278202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 456/895] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:34:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:34:58.279405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc769e-a48b-438a-9e72-77897c5483cb None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:58.291803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e59b684-a456-4ccd-9e96-80f5a7443c11 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:58.318339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:58.328595 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a2df7-4d33-4872-a2f8-b48b53304fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:58.331572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2df7-4d33-4872-a2f8-b48b53304fc3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:58.331572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2df7-4d33-4872-a2f8-b48b53304fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:58.331572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a2df7-4d33-4872-a2f8-b48b53304fc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:58.331572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a2df7-4d33-4872-a2f8-b48b53304fc3 None admin] RBAC: Adding query filter params (interface=public, service_id=e80a101f625e4384a2fd5d42b201b286, region_id=RegionOne) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:58.331572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a2df7-4d33-4872-a2f8-b48b53304fc3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:58.332064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a2df7-4d33-4872-a2f8-b48b53304fc3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:58.336853 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 440/896] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 19:34:58 2022] GET /identity/v3/endpoints?service_id=e80a101f625e4384a2fd5d42b201b286&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:58.337598 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a2df7-4d33-4872-a2f8-b48b53304fc3 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:58.345947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:58.390582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc769e-a48b-438a-9e72-77897c5483cb None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:58.426045 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a127a-7a54-4b98-83b4-0d193ff611b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:58.426651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a127a-7a54-4b98-83b4-0d193ff611b4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:58.426927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a127a-7a54-4b98-83b4-0d193ff611b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:58.427195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a127a-7a54-4b98-83b4-0d193ff611b4 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:58.427817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a127a-7a54-4b98-83b4-0d193ff611b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:58.428127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a127a-7a54-4b98-83b4-0d193ff611b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:58.429467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a127a-7a54-4b98-83b4-0d193ff611b4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:58.441720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 457/897] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:58.441720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a127a-7a54-4b98-83b4-0d193ff611b4 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:58.451930 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a2df7-4d33-4872-a2f8-b48b53304fc3 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:58.485578 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d85a3-83ae-49ca-bc84-9bc9e24c3a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:58.486200 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d85a3-83ae-49ca-bc84-9bc9e24c3a85 None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:58.486471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d85a3-83ae-49ca-bc84-9bc9e24c3a85 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:58.486739 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05d85a3-83ae-49ca-bc84-9bc9e24c3a85 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:58.487367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d85a3-83ae-49ca-bc84-9bc9e24c3a85 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:58.488882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05d85a3-83ae-49ca-bc84-9bc9e24c3a85 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:58.542226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a127a-7a54-4b98-83b4-0d193ff611b4 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:58.557954 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:58.569830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 441/898] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 19:34:58 2022] POST /identity/v3/services => generated 241 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:34:58.569830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d85a3-83ae-49ca-bc84-9bc9e24c3a85 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:58.583114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7aa630-31fd-4fa0-ad80-52923f97cf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:58.583742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa630-31fd-4fa0-ad80-52923f97cf4c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:58.584014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa630-31fd-4fa0-ad80-52923f97cf4c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:58.584276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7aa630-31fd-4fa0-ad80-52923f97cf4c None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:58.584882 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7aa630-31fd-4fa0-ad80-52923f97cf4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:58.586542 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7aa630-31fd-4fa0-ad80-52923f97cf4c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:58.598117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 458/899] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 19:34:58 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:34:58.600026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7aa630-31fd-4fa0-ad80-52923f97cf4c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:58.631219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:58.702898 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05d85a3-83ae-49ca-bc84-9bc9e24c3a85 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:58.750933 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589066b-e942-4fd9-b7a4-336e43d9eead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:58.751741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589066b-e942-4fd9-b7a4-336e43d9eead None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:58.752119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589066b-e942-4fd9-b7a4-336e43d9eead None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:58.752481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589066b-e942-4fd9-b7a4-336e43d9eead None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:58.757676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7aa630-31fd-4fa0-ad80-52923f97cf4c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:58.761838 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589066b-e942-4fd9-b7a4-336e43d9eead None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:58.764005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589066b-e942-4fd9-b7a4-336e43d9eead None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-5589066b-e942-4fd9-b7a4-336e43d9eead None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:34:58.779752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:58.780806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:58.781742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:58.781742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:58.781742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:34:58.781742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:34:58.781742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:34:58.781742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:34:58.781742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:34:58.781742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:34:58.781742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 19:34:58.781742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:58.785432 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 442/900] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 19:34:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:58.786748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589066b-e942-4fd9-b7a4-336e43d9eead None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:58.811284 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb610e-8ca5-40bf-9113-4fca85c732f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:58.811934 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb610e-8ca5-40bf-9113-4fca85c732f2 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:58.812212 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb610e-8ca5-40bf-9113-4fca85c732f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:58.812482 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb610e-8ca5-40bf-9113-4fca85c732f2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:58.813095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb610e-8ca5-40bf-9113-4fca85c732f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:58.815405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb610e-8ca5-40bf-9113-4fca85c732f2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-86cb610e-8ca5-40bf-9113-4fca85c732f2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:58.824959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:58.830147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:34:58.831572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 459/901] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Sep 14 19:34:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:58.832539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb610e-8ca5-40bf-9113-4fca85c732f2 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:58.877031 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:58.914347 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589066b-e942-4fd9-b7a4-336e43d9eead None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:58.929075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:58.939219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb610e-8ca5-40bf-9113-4fca85c732f2 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:58.961438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b7b96-a056-49f3-a077-ff7e9ab64f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:58.962831 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b7b96-a056-49f3-a077-ff7e9ab64f1f None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:58.963261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b7b96-a056-49f3-a077-ff7e9ab64f1f None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:58.963897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32b7b96-a056-49f3-a077-ff7e9ab64f1f None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:58.965871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b7b96-a056-49f3-a077-ff7e9ab64f1f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:58.966869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b7b96-a056-49f3-a077-ff7e9ab64f1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:58.969105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32b7b96-a056-49f3-a077-ff7e9ab64f1f None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:58.977069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8f1aa-bcb8-4bb6-a222-bcce7ce47413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:58.977676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8f1aa-bcb8-4bb6-a222-bcce7ce47413 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:58.978044 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8f1aa-bcb8-4bb6-a222-bcce7ce47413 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:58.978314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8f1aa-bcb8-4bb6-a222-bcce7ce47413 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:58.978931 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8f1aa-bcb8-4bb6-a222-bcce7ce47413 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:58.979258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8f1aa-bcb8-4bb6-a222-bcce7ce47413 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:58.980664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8f1aa-bcb8-4bb6-a222-bcce7ce47413 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:58.990975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 460/902] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 19:34:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:58.992013 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8f1aa-bcb8-4bb6-a222-bcce7ce47413 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:58.993673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 443/903] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Sep 14 19:34:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:59.001888 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b7b96-a056-49f3-a077-ff7e9ab64f1f None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:59.108587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8f1aa-bcb8-4bb6-a222-bcce7ce47413 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:59.111793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:59.137314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32b7b96-a056-49f3-a077-ff7e9ab64f1f None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:59.149191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ec7fc-8232-4908-92e5-5b482883e61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:59.149836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ec7fc-8232-4908-92e5-5b482883e61e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:59.150118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ec7fc-8232-4908-92e5-5b482883e61e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:59.150628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9ec7fc-8232-4908-92e5-5b482883e61e None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:59.151763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ec7fc-8232-4908-92e5-5b482883e61e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:59.152496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ec7fc-8232-4908-92e5-5b482883e61e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:59.154792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9ec7fc-8232-4908-92e5-5b482883e61e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:59.165148 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 461/904] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 19:34:58 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:34:59.166286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ec7fc-8232-4908-92e5-5b482883e61e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:59.179745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345760f9-7a56-4553-ac08-f8cb0af7012f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:59.180401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345760f9-7a56-4553-ac08-f8cb0af7012f None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:59.180684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345760f9-7a56-4553-ac08-f8cb0af7012f None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:59.180954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345760f9-7a56-4553-ac08-f8cb0af7012f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:59.220180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:59.263278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345760f9-7a56-4553-ac08-f8cb0af7012f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:59.265504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345760f9-7a56-4553-ac08-f8cb0af7012f None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:59.291027 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9ec7fc-8232-4908-92e5-5b482883e61e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:59.343151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbd05c-d62a-4042-85a6-66f606e2200e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:59.344052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbd05c-d62a-4042-85a6-66f606e2200e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:59.344496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbd05c-d62a-4042-85a6-66f606e2200e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:59.344914 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edbd05c-d62a-4042-85a6-66f606e2200e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:59.345757 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbd05c-d62a-4042-85a6-66f606e2200e None admin] RBAC: Adding query filter params (interface=public, service_id=7cadcca5c0024b4981cfe1c997a60c60, region_id=RegionOne) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:59.346236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbd05c-d62a-4042-85a6-66f606e2200e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:59.348316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edbd05c-d62a-4042-85a6-66f606e2200e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:59.357167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 462/905] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 19:34:59 2022] GET /identity/v3/endpoints?service_id=7cadcca5c0024b4981cfe1c997a60c60&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:59.362615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbd05c-d62a-4042-85a6-66f606e2200e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:59.381037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 444/906] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:34:59 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:34:59.387262 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345760f9-7a56-4553-ac08-f8cb0af7012f None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:59.406291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:59.459186 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:59.495295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345760f9-7a56-4553-ac08-f8cb0af7012f None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:59.505205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edbd05c-d62a-4042-85a6-66f606e2200e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:59.538876 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235f9db-3319-4344-a180-26a1973ef929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:59.540283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235f9db-3319-4344-a180-26a1973ef929 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:59.540965 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235f9db-3319-4344-a180-26a1973ef929 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:59.541426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8235f9db-3319-4344-a180-26a1973ef929 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:59.550439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9adaa6c-c717-426b-8c82-3e5154e0459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:59.554040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9adaa6c-c717-426b-8c82-3e5154e0459b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:59.554040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9adaa6c-c717-426b-8c82-3e5154e0459b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:59.554040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9adaa6c-c717-426b-8c82-3e5154e0459b None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:59.554040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9adaa6c-c717-426b-8c82-3e5154e0459b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:59.554040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9adaa6c-c717-426b-8c82-3e5154e0459b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:59.557634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235f9db-3319-4344-a180-26a1973ef929 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:59.560227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8235f9db-3319-4344-a180-26a1973ef929 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:59.566213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 463/907] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 19:34:59 2022] GET /identity/v3/services => generated 1497 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-8235f9db-3319-4344-a180-26a1973ef929 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:34:59.578296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:59.579597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:34:59.580719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:34:59.580719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:34:59.580719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:59.582653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 445/908] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:34:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:34:59.593961 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9adaa6c-c717-426b-8c82-3e5154e0459b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:59.696572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:59.703056 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9adaa6c-c717-426b-8c82-3e5154e0459b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:59.744753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcaa50-09a9-44c4-9374-c003b91160b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:59.746178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcaa50-09a9-44c4-9374-c003b91160b8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:59.746604 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcaa50-09a9-44c4-9374-c003b91160b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:59.747053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fcaa50-09a9-44c4-9374-c003b91160b8 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:59.747846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcaa50-09a9-44c4-9374-c003b91160b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:34:59.748295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcaa50-09a9-44c4-9374-c003b91160b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:59.749862 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fcaa50-09a9-44c4-9374-c003b91160b8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:59.763707 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 464/909] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:34:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:34:59.769062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235f9db-3319-4344-a180-26a1973ef929 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:34:59.875736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8235f9db-3319-4344-a180-26a1973ef929 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:34:59.897567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:34:59.912262 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26847ed6-9858-4593-8998-0d579b677c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:34:59.912995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26847ed6-9858-4593-8998-0d579b677c14 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:34:59.913537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26847ed6-9858-4593-8998-0d579b677c14 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:34:59.913674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26847ed6-9858-4593-8998-0d579b677c14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:34:59.921359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26847ed6-9858-4593-8998-0d579b677c14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:34:59.922901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26847ed6-9858-4593-8998-0d579b677c14 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-26847ed6-9858-4593-8998-0d579b677c14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:34:59.934872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:34:59.935782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:34:59.936741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:34:59.936741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:34:59.936741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:34:59.936741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:34:59.936741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:34:59.936741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:34:59.936741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:34:59.936741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:34:59.936741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 446/910] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:34:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:34:59.942428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcaa50-09a9-44c4-9374-c003b91160b8 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:00.046294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fcaa50-09a9-44c4-9374-c003b91160b8 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:00.081151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9154b0-64e3-44c5-8a48-28b071b6d25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:00.081758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9154b0-64e3-44c5-8a48-28b071b6d25c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:00.082067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9154b0-64e3-44c5-8a48-28b071b6d25c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:00.082334 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9154b0-64e3-44c5-8a48-28b071b6d25c None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:00.083006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9154b0-64e3-44c5-8a48-28b071b6d25c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:00.083303 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9154b0-64e3-44c5-8a48-28b071b6d25c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:00.084704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9154b0-64e3-44c5-8a48-28b071b6d25c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:00.096910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 465/911] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:34:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:00.102807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26847ed6-9858-4593-8998-0d579b677c14 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:00.209179 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26847ed6-9858-4593-8998-0d579b677c14 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:00.245282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c7e31e-95b9-416a-919d-03cb939307f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:00.246811 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7e31e-95b9-416a-919d-03cb939307f5 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:00.247308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7e31e-95b9-416a-919d-03cb939307f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:00.247807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c7e31e-95b9-416a-919d-03cb939307f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:00.248736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7e31e-95b9-416a-919d-03cb939307f5 None admin] RBAC: Adding query filter params (role.id=688c3a60f8d34392878a70a5abaffefe, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=7d5605f645a74b1b842733dda256193c) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:00.249236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7e31e-95b9-416a-919d-03cb939307f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:00.250886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c7e31e-95b9-416a-919d-03cb939307f5 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:00.257118 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 447/912] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:35:00 2022] GET /identity/v3/role_assignments?user.id=7d5605f645a74b1b842733dda256193c&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=688c3a60f8d34392878a70a5abaffefe => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:00.816142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620f01d-54d8-444b-897d-d948100a5afd None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:00.816666 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620f01d-54d8-444b-897d-d948100a5afd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:00.816666 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620f01d-54d8-444b-897d-d948100a5afd None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:00.818558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 466/913] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:00.823900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93982b67-ed28-4241-adc4-b41432e4ae82 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:00.824187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93982b67-ed28-4241-adc4-b41432e4ae82 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:00.824464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93982b67-ed28-4241-adc4-b41432e4ae82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:00.834591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfa625-cc61-4f18-bdca-f9fd4f5c1670 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:00.834873 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfa625-cc61-4f18-bdca-f9fd4f5c1670 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:00.835175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfa625-cc61-4f18-bdca-f9fd4f5c1670 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:00.836711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 467/914] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:35:00.841533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368388f8-68d1-425d-8fc9-8443c04a9701 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:00.841828 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368388f8-68d1-425d-8fc9-8443c04a9701 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:00.842110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368388f8-68d1-425d-8fc9-8443c04a9701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:00.843555 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-93982b67-ed28-4241-adc4-b41432e4ae82 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:00.844745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-93982b67-ed28-4241-adc4-b41432e4ae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:00.862625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-368388f8-68d1-425d-8fc9-8443c04a9701 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:00.863803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-368388f8-68d1-425d-8fc9-8443c04a9701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:00.877496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:00.903362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:00.934456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 448/915] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:00.941558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 468/916] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:35:01.207697 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93982b67-ed28-4241-adc4-b41432e4ae82 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:01.217431 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368388f8-68d1-425d-8fc9-8443c04a9701 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:01.324424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93982b67-ed28-4241-adc4-b41432e4ae82 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:01.330031 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368388f8-68d1-425d-8fc9-8443c04a9701 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:01.366403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972940af-786d-41ea-b9e3-42b343e42d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:01.367120 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972940af-786d-41ea-b9e3-42b343e42d80 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:01.367470 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972940af-786d-41ea-b9e3-42b343e42d80 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:01.367806 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972940af-786d-41ea-b9e3-42b343e42d80 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:01.368451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972940af-786d-41ea-b9e3-42b343e42d80 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:01.369911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972940af-786d-41ea-b9e3-42b343e42d80 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:01.370557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b8a14-55b2-4709-bd76-5407fa7ef385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:01.371344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b8a14-55b2-4709-bd76-5407fa7ef385 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:01.371761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b8a14-55b2-4709-bd76-5407fa7ef385 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:01.372427 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b8a14-55b2-4709-bd76-5407fa7ef385 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:01.373566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b8a14-55b2-4709-bd76-5407fa7ef385 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:01.375395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4b8a14-55b2-4709-bd76-5407fa7ef385 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-972940af-786d-41ea-b9e3-42b343e42d80 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:01.379975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:01.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:01.382775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:01.382775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:01.382775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:01.382775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:01.382775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:01.382775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:01.382775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:01.382775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:01.382775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:01.382775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:01.382775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:35:01.382775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:01.382775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 19:35:01.382775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:01.383524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 469/917] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:35:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-bf4b8a14-55b2-4709-bd76-5407fa7ef385 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:01.389367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:01.391657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:01.393865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 449/918] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 19:35:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:01.395824 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972940af-786d-41ea-b9e3-42b343e42d80 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:01.395983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b8a14-55b2-4709-bd76-5407fa7ef385 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:01.410682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:01.440500 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:01.510624 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972940af-786d-41ea-b9e3-42b343e42d80 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:01.512724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4b8a14-55b2-4709-bd76-5407fa7ef385 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:01.548203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92ca32-e51a-46ea-8c69-374c8cb2f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:01.548844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92ca32-e51a-46ea-8c69-374c8cb2f774 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:01.549121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92ca32-e51a-46ea-8c69-374c8cb2f774 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:01.549392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e92ca32-e51a-46ea-8c69-374c8cb2f774 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:01.550046 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92ca32-e51a-46ea-8c69-374c8cb2f774 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:01.550346 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92ca32-e51a-46ea-8c69-374c8cb2f774 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:01.551750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e92ca32-e51a-46ea-8c69-374c8cb2f774 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:01.553094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216cb2e-e090-4b24-bf00-5421a4c4b6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:01.553826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216cb2e-e090-4b24-bf00-5421a4c4b6eb None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:01.554131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216cb2e-e090-4b24-bf00-5421a4c4b6eb None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:01.554407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216cb2e-e090-4b24-bf00-5421a4c4b6eb None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:01.555088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216cb2e-e090-4b24-bf00-5421a4c4b6eb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:01.555449 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216cb2e-e090-4b24-bf00-5421a4c4b6eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:01.556824 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216cb2e-e090-4b24-bf00-5421a4c4b6eb None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:01.557283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 470/919] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:35:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:01.562830 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92ca32-e51a-46ea-8c69-374c8cb2f774 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:01.565871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 450/920] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 19:35:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:01.568343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216cb2e-e090-4b24-bf00-5421a4c4b6eb None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:01.648993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:01.675189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e92ca32-e51a-46ea-8c69-374c8cb2f774 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:01.701042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216cb2e-e090-4b24-bf00-5421a4c4b6eb None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:01.703040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:01.722223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b2035-67d0-4f07-9ac9-309cfde06a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:01.722976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b2035-67d0-4f07-9ac9-309cfde06a51 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:01.723383 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b2035-67d0-4f07-9ac9-309cfde06a51 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:01.723660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b2035-67d0-4f07-9ac9-309cfde06a51 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:01.724275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b2035-67d0-4f07-9ac9-309cfde06a51 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:01.724571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b2035-67d0-4f07-9ac9-309cfde06a51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:01.726453 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b2035-67d0-4f07-9ac9-309cfde06a51 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:01.733838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 471/921] 10.222.0.42 () {56 vars in 1114 bytes} [Wed Sep 14 19:35:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:01.738536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b2035-67d0-4f07-9ac9-309cfde06a51 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:01.742314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0ea4d-4a22-49a3-8b1d-8a1dcd3c8d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:01.742817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0ea4d-4a22-49a3-8b1d-8a1dcd3c8d80 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:01.743088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0ea4d-4a22-49a3-8b1d-8a1dcd3c8d80 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:01.743352 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da0ea4d-4a22-49a3-8b1d-8a1dcd3c8d80 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:01.743986 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0ea4d-4a22-49a3-8b1d-8a1dcd3c8d80 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:01.744956 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0ea4d-4a22-49a3-8b1d-8a1dcd3c8d80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:01.746218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da0ea4d-4a22-49a3-8b1d-8a1dcd3c8d80 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:01.756357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 451/922] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 19:35:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:01.757279 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0ea4d-4a22-49a3-8b1d-8a1dcd3c8d80 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:01.855551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b2035-67d0-4f07-9ac9-309cfde06a51 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:01.880517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da0ea4d-4a22-49a3-8b1d-8a1dcd3c8d80 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:01.896223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123d114-08ed-4820-8d12-eaa3d943014d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:01.896830 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123d114-08ed-4820-8d12-eaa3d943014d None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:01.897107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123d114-08ed-4820-8d12-eaa3d943014d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:01.897424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123d114-08ed-4820-8d12-eaa3d943014d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:01.898056 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123d114-08ed-4820-8d12-eaa3d943014d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:01.899522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123d114-08ed-4820-8d12-eaa3d943014d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:01.910024 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:01.918635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc48ec-2340-4d09-9f0b-a2474c4dddc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:01.919354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc48ec-2340-4d09-9f0b-a2474c4dddc9 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:01.920638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc48ec-2340-4d09-9f0b-a2474c4dddc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:01.920638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc48ec-2340-4d09-9f0b-a2474c4dddc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:01.920638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc48ec-2340-4d09-9f0b-a2474c4dddc9 None admin] RBAC: Adding query filter params (interface=public, service_id=542944ae2e50451f9e3d5201e5f9d834, region_id=RegionOne) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:01.921040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc48ec-2340-4d09-9f0b-a2474c4dddc9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:01.922494 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc48ec-2340-4d09-9f0b-a2474c4dddc9 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:01.927830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 452/923] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 19:35:01 2022] GET /identity/v3/endpoints?service_id=542944ae2e50451f9e3d5201e5f9d834&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:01.933944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc48ec-2340-4d09-9f0b-a2474c4dddc9 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:01.964970 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:01.984658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 472/924] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:35:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:35:01.984658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5868d3a-9a7e-438d-9794-1fc1ec2b7ff9 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:01.984658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5868d3a-9a7e-438d-9794-1fc1ec2b7ff9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:01.984658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5868d3a-9a7e-438d-9794-1fc1ec2b7ff9 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:01.984658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 473/925] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:01.988700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b3c32-6fc0-4a57-aadc-aeede514a1c6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:01.989108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b3c32-6fc0-4a57-aadc-aeede514a1c6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:01.989454 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24b3c32-6fc0-4a57-aadc-aeede514a1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:02.013301 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b24b3c32-6fc0-4a57-aadc-aeede514a1c6 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:02.014698 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b24b3c32-6fc0-4a57-aadc-aeede514a1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:02.054465 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc48ec-2340-4d09-9f0b-a2474c4dddc9 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:02.103591 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c489585-fcc9-41b9-9f94-3681665a64a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:02.104118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 474/926] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:35:02.106612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c489585-fcc9-41b9-9f94-3681665a64a4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:02.107077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c489585-fcc9-41b9-9f94-3681665a64a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:02.107543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c489585-fcc9-41b9-9f94-3681665a64a4 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:02.108369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c489585-fcc9-41b9-9f94-3681665a64a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:02.109964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c489585-fcc9-41b9-9f94-3681665a64a4 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:02.117334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 453/927] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 19:35:01 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:35:02.388762 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b3c32-6fc0-4a57-aadc-aeede514a1c6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:02.498158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24b3c32-6fc0-4a57-aadc-aeede514a1c6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:02.512070 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e877fa-92b5-4723-a949-521f72b478b3 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:02.512606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e877fa-92b5-4723-a949-521f72b478b3 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:02.512606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e877fa-92b5-4723-a949-521f72b478b3 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:02.514220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 454/928] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:35:02.519077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72b732-3fe5-4354-8bb7-421606f70f8a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:02.519355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72b732-3fe5-4354-8bb7-421606f70f8a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:02.519627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72b732-3fe5-4354-8bb7-421606f70f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:02.530615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:02.540137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-ec72b732-3fe5-4354-8bb7-421606f70f8a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:02.541333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ec72b732-3fe5-4354-8bb7-421606f70f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:02.542211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d73f5-4f2a-4c85-9a82-725afc902e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:02.543053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d73f5-4f2a-4c85-9a82-725afc902e0d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:02.543573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d73f5-4f2a-4c85-9a82-725afc902e0d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:02.544050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d73f5-4f2a-4c85-9a82-725afc902e0d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:02.544912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d73f5-4f2a-4c85-9a82-725afc902e0d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:02.546499 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d73f5-4f2a-4c85-9a82-725afc902e0d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-a28d73f5-4f2a-4c85-9a82-725afc902e0d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:02.556308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:02.557450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:02.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:02.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:02.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:02.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:02.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:02.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:02.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:02.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:02.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:02.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:02.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:35:02.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:02.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 19:35:02.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:02.561717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 475/929] 10.222.0.42 () {56 vars in 1132 bytes} [Wed Sep 14 19:35:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:02.575369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d73f5-4f2a-4c85-9a82-725afc902e0d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:02.622104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 455/930] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:35:02.688613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d73f5-4f2a-4c85-9a82-725afc902e0d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:02.722374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7763e-ecc0-4086-b208-8b25feb6ab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:02.722982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7763e-ecc0-4086-b208-8b25feb6ab44 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:02.723260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7763e-ecc0-4086-b208-8b25feb6ab44 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:02.723529 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea7763e-ecc0-4086-b208-8b25feb6ab44 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:02.724148 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7763e-ecc0-4086-b208-8b25feb6ab44 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:02.724449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7763e-ecc0-4086-b208-8b25feb6ab44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:02.725816 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea7763e-ecc0-4086-b208-8b25feb6ab44 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:02.730848 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 476/931] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 19:35:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:02.736574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72b732-3fe5-4354-8bb7-421606f70f8a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:02.759327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:02.843395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72b732-3fe5-4354-8bb7-421606f70f8a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:02.884137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1911e-cf40-46b6-8bf9-67b027980e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:02.884756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1911e-cf40-46b6-8bf9-67b027980e49 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:02.885037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1911e-cf40-46b6-8bf9-67b027980e49 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:02.885314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1911e-cf40-46b6-8bf9-67b027980e49 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:02.886008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1911e-cf40-46b6-8bf9-67b027980e49 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:02.886317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1911e-cf40-46b6-8bf9-67b027980e49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:02.887759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca1911e-cf40-46b6-8bf9-67b027980e49 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:02.892989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 456/932] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 19:35:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:02.898599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7763e-ecc0-4086-b208-8b25feb6ab44 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:02.927668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1911e-cf40-46b6-8bf9-67b027980e49 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:02.950960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:03.003918 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea7763e-ecc0-4086-b208-8b25feb6ab44 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:03.034198 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:03.038616 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca1911e-cf40-46b6-8bf9-67b027980e49 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:03.039475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcce303-9624-4914-ad9c-16a70f72b81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:03.040107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcce303-9624-4914-ad9c-16a70f72b81d None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:03.040383 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcce303-9624-4914-ad9c-16a70f72b81d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:03.040653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcce303-9624-4914-ad9c-16a70f72b81d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:03.041345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcce303-9624-4914-ad9c-16a70f72b81d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:03.043335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcce303-9624-4914-ad9c-16a70f72b81d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:03.077566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71524d6-93da-4fd3-94f3-194becbe7ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:03.078306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71524d6-93da-4fd3-94f3-194becbe7ec2 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:03.078647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71524d6-93da-4fd3-94f3-194becbe7ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:03.078983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71524d6-93da-4fd3-94f3-194becbe7ec2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:03.090845 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71524d6-93da-4fd3-94f3-194becbe7ec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-a71524d6-93da-4fd3-94f3-194becbe7ec2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:35:03.095753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:03.096735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:03.097634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:03.097634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:03.097634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:03.097634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:03.097634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:35:03.097634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:35:03.097634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:35:03.097634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:35:03.097634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:35:03.097634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:35:03.097634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 19:35:03.097634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:03.097634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 457/933] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Sep 14 19:35:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:03.103984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71524d6-93da-4fd3-94f3-194becbe7ec2 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:03.148946 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 477/934] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:35:02 2022] POST /identity/v3/endpoints => generated 347 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:35:03.226068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71524d6-93da-4fd3-94f3-194becbe7ec2 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:03.271917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0efa46-570b-46b3-b644-07c85e265ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:03.272815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0efa46-570b-46b3-b644-07c85e265ce9 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:03.273314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0efa46-570b-46b3-b644-07c85e265ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:03.273726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0efa46-570b-46b3-b644-07c85e265ce9 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:03.274576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0efa46-570b-46b3-b644-07c85e265ce9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:03.275025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0efa46-570b-46b3-b644-07c85e265ce9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:03.277130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0efa46-570b-46b3-b644-07c85e265ce9 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:03.287492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 458/935] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Sep 14 19:35:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:03.297886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcce303-9624-4914-ad9c-16a70f72b81d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:03.407901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcce303-9624-4914-ad9c-16a70f72b81d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:03.411196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:03.443571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf91f6c-90f2-4923-b852-d5a9a5a7e9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:03.444203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf91f6c-90f2-4923-b852-d5a9a5a7e9ae None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:03.444478 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf91f6c-90f2-4923-b852-d5a9a5a7e9ae None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:03.444748 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf91f6c-90f2-4923-b852-d5a9a5a7e9ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:03.456589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf91f6c-90f2-4923-b852-d5a9a5a7e9ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:03.458067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf91f6c-90f2-4923-b852-d5a9a5a7e9ae None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-9bf91f6c-90f2-4923-b852-d5a9a5a7e9ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:35:03.473460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:03.474434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:35:03.475322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:35:03.475322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:03.475322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:03.475322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 478/936] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:03.483084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0efa46-570b-46b3-b644-07c85e265ce9 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:03.486581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:03.594779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0efa46-570b-46b3-b644-07c85e265ce9 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:03.630118 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e020a53-ef87-438d-95e7-f3aeb15d1d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:03.630775 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e020a53-ef87-438d-95e7-f3aeb15d1d8a None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:03.631052 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e020a53-ef87-438d-95e7-f3aeb15d1d8a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:03.631375 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e020a53-ef87-438d-95e7-f3aeb15d1d8a None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:03.632013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e020a53-ef87-438d-95e7-f3aeb15d1d8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:03.632314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e020a53-ef87-438d-95e7-f3aeb15d1d8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:03.633722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e020a53-ef87-438d-95e7-f3aeb15d1d8a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:03.646477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 459/937] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:35:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:03.654876 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf91f6c-90f2-4923-b852-d5a9a5a7e9ae None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:03.763679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf91f6c-90f2-4923-b852-d5a9a5a7e9ae None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:03.800370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123dd4b-b8df-4b46-80fd-4ec298657321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:03.801370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123dd4b-b8df-4b46-80fd-4ec298657321 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:03.801871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123dd4b-b8df-4b46-80fd-4ec298657321 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:03.802357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8123dd4b-b8df-4b46-80fd-4ec298657321 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:03.810626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123dd4b-b8df-4b46-80fd-4ec298657321 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:03.812302 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8123dd4b-b8df-4b46-80fd-4ec298657321 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-8123dd4b-b8df-4b46-80fd-4ec298657321 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:35:03.825140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:03.826375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:03.827307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:03.827307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:03.827307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:35:03.827307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:35:03.827307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:35:03.827307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:35:03.827307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:35:03.827307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:35:03.827307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 19:35:03.827307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:03.829622 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 479/938] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 19:35:03 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:03.835637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e020a53-ef87-438d-95e7-f3aeb15d1d8a None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:03.907743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:03.948264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e020a53-ef87-438d-95e7-f3aeb15d1d8a None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:03.988371 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49219719-3fa4-499c-b15b-86e3600e96ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:03.988984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49219719-3fa4-499c-b15b-86e3600e96ed None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:03.989262 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49219719-3fa4-499c-b15b-86e3600e96ed None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:03.989530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49219719-3fa4-499c-b15b-86e3600e96ed None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:03.990299 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49219719-3fa4-499c-b15b-86e3600e96ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:03.990600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49219719-3fa4-499c-b15b-86e3600e96ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:03.992015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49219719-3fa4-499c-b15b-86e3600e96ed None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:04.010153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 460/939] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Sep 14 19:35:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:04.015877 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123dd4b-b8df-4b46-80fd-4ec298657321 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:04.036315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:04.123766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8123dd4b-b8df-4b46-80fd-4ec298657321 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:04.163118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53692e-ff26-4bd9-813d-cf6afe4a63b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:04.164158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53692e-ff26-4bd9-813d-cf6afe4a63b0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:04.164674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53692e-ff26-4bd9-813d-cf6afe4a63b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:04.165141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53692e-ff26-4bd9-813d-cf6afe4a63b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:04.178712 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53692e-ff26-4bd9-813d-cf6afe4a63b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:04.180617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53692e-ff26-4bd9-813d-cf6afe4a63b0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-6a53692e-ff26-4bd9-813d-cf6afe4a63b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:35:04.197242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:04.198255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:35:04.199196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:35:04.199196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:04.199196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:04.201283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 480/940] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:04.207261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49219719-3fa4-499c-b15b-86e3600e96ed None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:04.280916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168af47e-a8a8-44b2-af86-e79750328787 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:04.281105 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168af47e-a8a8-44b2-af86-e79750328787 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:04.281379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168af47e-a8a8-44b2-af86-e79750328787 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:04.283027 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 481/941] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:35:04.288083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bed93-3c7f-4718-a5e3-e9f9944b89d0 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:04.288366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bed93-3c7f-4718-a5e3-e9f9944b89d0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:04.288643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05bed93-3c7f-4718-a5e3-e9f9944b89d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:04.312943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e05bed93-3c7f-4718-a5e3-e9f9944b89d0 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:04.314499 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e05bed93-3c7f-4718-a5e3-e9f9944b89d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:04.315113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49219719-3fa4-499c-b15b-86e3600e96ed None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:04.336744 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:04.351429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43551039-9b07-4ad7-95c3-0eb0529d35e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:04.352027 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43551039-9b07-4ad7-95c3-0eb0529d35e6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:04.374072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43551039-9b07-4ad7-95c3-0eb0529d35e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:04.374072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43551039-9b07-4ad7-95c3-0eb0529d35e6 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:04.374072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43551039-9b07-4ad7-95c3-0eb0529d35e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:04.374072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43551039-9b07-4ad7-95c3-0eb0529d35e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:04.374072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43551039-9b07-4ad7-95c3-0eb0529d35e6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:04.374072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 461/942] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:35:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:04.374517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c378e-3b78-41f6-a559-ba9e29d082b5 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:04.374517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c378e-3b78-41f6-a559-ba9e29d082b5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:04.374517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c378e-3b78-41f6-a559-ba9e29d082b5 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:04.374517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 462/943] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:04.374968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c378e-3b78-41f6-a559-ba9e29d082b5 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:04.389917 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:04.396418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 482/944] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:35:04.397475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb4d53-b633-4d26-b859-6a91e109b446 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:04.397760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb4d53-b633-4d26-b859-6a91e109b446 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:04.398073 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb4d53-b633-4d26-b859-6a91e109b446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:04.418037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4cbb4d53-b633-4d26-b859-6a91e109b446 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:04.420289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4cbb4d53-b633-4d26-b859-6a91e109b446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:04.499494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 483/945] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:04.500726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c378e-3b78-41f6-a559-ba9e29d082b5 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:04.539125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf911f65-3da9-4856-b136-ecde25a66eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:04.539685 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf911f65-3da9-4856-b136-ecde25a66eed None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:04.539997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf911f65-3da9-4856-b136-ecde25a66eed None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:04.540266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf911f65-3da9-4856-b136-ecde25a66eed None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:04.547397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf911f65-3da9-4856-b136-ecde25a66eed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:04.548906 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf911f65-3da9-4856-b136-ecde25a66eed None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-bf911f65-3da9-4856-b136-ecde25a66eed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:35:04.560722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:04.563119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:04.564041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:35:04.564041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:35:04.564041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:35:04.564041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:35:04.564041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:35:04.564041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:35:04.564041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:35:04.564041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:04.564041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 463/946] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:35:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:04.572238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb4d53-b633-4d26-b859-6a91e109b446 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:04.668802 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf911f65-3da9-4856-b136-ecde25a66eed None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:04.676820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb4d53-b633-4d26-b859-6a91e109b446 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:04.715091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:04.722586 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:04.728512 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7014fd1-fc4a-4dc2-bc16-6c8aeec3b4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:04.729751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7014fd1-fc4a-4dc2-bc16-6c8aeec3b4a6 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:04.730304 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7014fd1-fc4a-4dc2-bc16-6c8aeec3b4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:04.730930 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7014fd1-fc4a-4dc2-bc16-6c8aeec3b4a6 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:04.732106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7014fd1-fc4a-4dc2-bc16-6c8aeec3b4a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:04.732617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7014fd1-fc4a-4dc2-bc16-6c8aeec3b4a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:04.734712 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7014fd1-fc4a-4dc2-bc16-6c8aeec3b4a6 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:04.750642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 484/947] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:35:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:04.756825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7014fd1-fc4a-4dc2-bc16-6c8aeec3b4a6 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:04.795592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf911f65-3da9-4856-b136-ecde25a66eed None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:04.833128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a22a5-0d1b-4161-a884-56bbe8d81c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:04.833725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a22a5-0d1b-4161-a884-56bbe8d81c60 None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:04.834036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a22a5-0d1b-4161-a884-56bbe8d81c60 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:04.834300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292a22a5-0d1b-4161-a884-56bbe8d81c60 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:04.834940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a22a5-0d1b-4161-a884-56bbe8d81c60 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:04.836419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292a22a5-0d1b-4161-a884-56bbe8d81c60 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:04.839062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: WARNING keystone.server.flask.common [None req-292a22a5-0d1b-4161-a884-56bbe8d81c60 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 14 19:35:04.869682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7014fd1-fc4a-4dc2-bc16-6c8aeec3b4a6 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:04.877132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.sql.core [None req-292a22a5-0d1b-4161-a884-56bbe8d81c60 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 19:35:04.877132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 14 19:35:04.877132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [parameters: {'id': 'f2f565c866ba4d1e8530e4613a7181bb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 14 19:35:04.877132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93426) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 19:35:04.885426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-292a22a5-0d1b-4161-a884-56bbe8d81c60 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f2f565c866ba4d1e8530e4613a7181bb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f2f565c866ba4d1e8530e4613a7181bb. Sep 14 19:35:04.885426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:04.885426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 19:35:04.885426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 19:35:04.885426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 19:35:04.885426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 19:35:04.885426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 19:35:04.885426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 19:35:04.885426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 19:35:04.885426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application conn.query(q) Sep 14 19:35:04.885426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 19:35:04.885426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 19:35:04.885426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application result.read() Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 14 19:35:04.886949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application next(self.gen) Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application yield resource Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application next(self.gen) Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise self.value Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application session.commit() Sep 14 19:35:04.887935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.session.flush() Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application compat.raise_( Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 19:35:04.888843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 19:35:04.889756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application conn.query(q) Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application result.read() Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 19:35:04.890581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [parameters: {'id': 'f2f565c866ba4d1e8530e4613a7181bb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:04.893225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:04.896566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:04.899148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:04.899148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 14 19:35:04.899148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 14 19:35:04.899148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:04.899148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:04.899148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 14 19:35:04.899148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 14 19:35:04.899148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f2f565c866ba4d1e8530e4613a7181bb. Sep 14 19:35:04.899148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:04.899148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 464/948] 10.222.0.42 () {60 vars in 1140 bytes} [Wed Sep 14 19:35:04 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:04.899148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a22a5-0d1b-4161-a884-56bbe8d81c60 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:04.914068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cfa32-8804-4e7a-a578-a60ec30849e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:04.914864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cfa32-8804-4e7a-a578-a60ec30849e7 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:04.915136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cfa32-8804-4e7a-a578-a60ec30849e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:04.915401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76cfa32-8804-4e7a-a578-a60ec30849e7 None admin] PATH_INFO: `/v3/projects/6bfc6c1c443849038af35577daf34088/users/7d5605f645a74b1b842733dda256193c/roles/688c3a60f8d34392878a70a5abaffefe` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:04.918805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76cfa32-8804-4e7a-a578-a60ec30849e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bfc6c1c443849038af35577daf34088, user_id=7d5605f645a74b1b842733dda256193c, role_id=688c3a60f8d34392878a70a5abaffefe)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:04.920610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76cfa32-8804-4e7a-a578-a60ec30849e7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:04.982646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 485/949] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:35:04 2022] PUT /identity/v3/projects/6bfc6c1c443849038af35577daf34088/users/7d5605f645a74b1b842733dda256193c/roles/688c3a60f8d34392878a70a5abaffefe => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:35:04.995724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cfa32-8804-4e7a-a578-a60ec30849e7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:04.999586 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:05.006196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292a22a5-0d1b-4161-a884-56bbe8d81c60 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:05.045346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829100b-92d5-4295-abb4-9011459fa38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:05.052508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829100b-92d5-4295-abb4-9011459fa38b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:05.052508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829100b-92d5-4295-abb4-9011459fa38b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:05.052508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f829100b-92d5-4295-abb4-9011459fa38b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:05.052508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829100b-92d5-4295-abb4-9011459fa38b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:05.052508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f829100b-92d5-4295-abb4-9011459fa38b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-f829100b-92d5-4295-abb4-9011459fa38b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:05.066403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:05.067957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:05.068992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:05.068992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:05.068992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:05.068992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:05.068992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:05.068992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:05.068992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:05.068992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:05.068992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:05.068992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:05.068992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:35:05.068992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:05.068992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:35:05.068992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:05.076395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 465/950] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 19:35:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:05.077550 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829100b-92d5-4295-abb4-9011459fa38b None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:05.092498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:05.163220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76cfa32-8804-4e7a-a578-a60ec30849e7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:05.206855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88241bca-f136-4c4b-89dd-05b3ce80ba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:05.207727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88241bca-f136-4c4b-89dd-05b3ce80ba3b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:05.208160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88241bca-f136-4c4b-89dd-05b3ce80ba3b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:05.208590 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88241bca-f136-4c4b-89dd-05b3ce80ba3b None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:05.211298 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f829100b-92d5-4295-abb4-9011459fa38b None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:05.216361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88241bca-f136-4c4b-89dd-05b3ce80ba3b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:05.217433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88241bca-f136-4c4b-89dd-05b3ce80ba3b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-88241bca-f136-4c4b-89dd-05b3ce80ba3b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:35:05.230683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:05.231895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:05.232851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:35:05.232851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:35:05.232851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:35:05.232851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:35:05.232851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:35:05.232851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:35:05.232851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:35:05.232851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:05.232851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 486/951] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:35:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:05.238298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88241bca-f136-4c4b-89dd-05b3ce80ba3b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:05.259977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225bd24-4573-475a-b9eb-dd96e7f2a7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:05.260587 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225bd24-4573-475a-b9eb-dd96e7f2a7bd None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:05.261385 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225bd24-4573-475a-b9eb-dd96e7f2a7bd None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:05.261667 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225bd24-4573-475a-b9eb-dd96e7f2a7bd None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:05.262350 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225bd24-4573-475a-b9eb-dd96e7f2a7bd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:05.262646 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225bd24-4573-475a-b9eb-dd96e7f2a7bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:05.264545 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225bd24-4573-475a-b9eb-dd96e7f2a7bd None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:05.275548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 466/952] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 19:35:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:05.280900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225bd24-4573-475a-b9eb-dd96e7f2a7bd None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:05.314294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:05.353128 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88241bca-f136-4c4b-89dd-05b3ce80ba3b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:05.362945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:05.387996 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d915aa-1311-4d21-a83e-6cc60d4bc42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:05.388843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d915aa-1311-4d21-a83e-6cc60d4bc42b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:05.389219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d915aa-1311-4d21-a83e-6cc60d4bc42b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:05.389584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d915aa-1311-4d21-a83e-6cc60d4bc42b None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:05.390398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d915aa-1311-4d21-a83e-6cc60d4bc42b None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:05.390800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d915aa-1311-4d21-a83e-6cc60d4bc42b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:05.392460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d915aa-1311-4d21-a83e-6cc60d4bc42b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:05.402273 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225bd24-4573-475a-b9eb-dd96e7f2a7bd None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:05.413851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 487/953] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 19:35:05 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:05.448216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0b1e5-c114-4ad4-a4ba-fa3a225280a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:05.449270 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0b1e5-c114-4ad4-a4ba-fa3a225280a4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:05.449746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0b1e5-c114-4ad4-a4ba-fa3a225280a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:05.450236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0b1e5-c114-4ad4-a4ba-fa3a225280a4 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:05.451089 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0b1e5-c114-4ad4-a4ba-fa3a225280a4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:05.451612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0b1e5-c114-4ad4-a4ba-fa3a225280a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:05.453208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0b1e5-c114-4ad4-a4ba-fa3a225280a4 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:05.459441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 467/954] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 19:35:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:05.466996 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d915aa-1311-4d21-a83e-6cc60d4bc42b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:05.523490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e96d57-270e-43bc-8bc0-27ec4f78569e None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:05.523867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e96d57-270e-43bc-8bc0-27ec4f78569e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:05.524150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e96d57-270e-43bc-8bc0-27ec4f78569e None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:05.527719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 468/955] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:05.532535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6e755-8198-4c2c-b571-08603085c031 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:05.532821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6e755-8198-4c2c-b571-08603085c031 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:05.533495 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6e755-8198-4c2c-b571-08603085c031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:05.571304 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-36e6e755-8198-4c2c-b571-08603085c031 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:05.572783 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-36e6e755-8198-4c2c-b571-08603085c031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:05.596523 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d915aa-1311-4d21-a83e-6cc60d4bc42b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:05.638317 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a530efd-011b-4e5f-970f-0813bd186c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:05.639227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a530efd-011b-4e5f-970f-0813bd186c22 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:05.639742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a530efd-011b-4e5f-970f-0813bd186c22 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:05.640111 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a530efd-011b-4e5f-970f-0813bd186c22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:05.640875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a530efd-011b-4e5f-970f-0813bd186c22 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:05.642393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a530efd-011b-4e5f-970f-0813bd186c22 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:05.650349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 469/956] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:05.731270 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 488/957] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:35:05 2022] POST /identity/v3/endpoints => generated 349 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:35:05.919710 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6e755-8198-4c2c-b571-08603085c031 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:05.939690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:06.029156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6e755-8198-4c2c-b571-08603085c031 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:06.071745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7f0de-1155-4528-9e06-d7f3a50b3beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:06.072617 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7f0de-1155-4528-9e06-d7f3a50b3beb None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:06.073085 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7f0de-1155-4528-9e06-d7f3a50b3beb None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:06.073528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b7f0de-1155-4528-9e06-d7f3a50b3beb None admin] PATH_INFO: `/v3/services/compute` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:06.074348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7f0de-1155-4528-9e06-d7f3a50b3beb None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:06.075919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b7f0de-1155-4528-9e06-d7f3a50b3beb None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-50b7f0de-1155-4528-9e06-d7f3a50b3beb None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:06.085049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:06.086335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:06.087390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:06.087390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:06.087390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:06.087390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:06.087390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:06.087390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:06.087390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:06.087390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:06.087390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:06.087390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:06.087390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:35:06.087390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:06.087390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 19:35:06.087390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:06.093898 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 470/958] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:35:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:06.095998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a530efd-011b-4e5f-970f-0813bd186c22 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:06.123261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:06.205715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a530efd-011b-4e5f-970f-0813bd186c22 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:06.240401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bdfa1a-2277-4f00-a108-ee697f207502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:06.241286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdfa1a-2277-4f00-a108-ee697f207502 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:06.241711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdfa1a-2277-4f00-a108-ee697f207502 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:06.242160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bdfa1a-2277-4f00-a108-ee697f207502 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:06.242927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdfa1a-2277-4f00-a108-ee697f207502 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:06.243390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdfa1a-2277-4f00-a108-ee697f207502 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:06.244889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bdfa1a-2277-4f00-a108-ee697f207502 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:06.250807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 489/959] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 19:35:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:06.259407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7f0de-1155-4528-9e06-d7f3a50b3beb None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:06.339428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:06.361083 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b7f0de-1155-4528-9e06-d7f3a50b3beb None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:06.396995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a2934-d408-4582-8170-a2e3dede47f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:06.397522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a2934-d408-4582-8170-a2e3dede47f8 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:06.397821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a2934-d408-4582-8170-a2e3dede47f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:06.398123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a2934-d408-4582-8170-a2e3dede47f8 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:06.398748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a2934-d408-4582-8170-a2e3dede47f8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:06.399048 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a2934-d408-4582-8170-a2e3dede47f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:06.400409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a2934-d408-4582-8170-a2e3dede47f8 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:06.405394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 471/960] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 19:35:06 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:07.268317 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa36950-c068-4e07-8df6-a22a9d6de69e None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:07.268831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa36950-c068-4e07-8df6-a22a9d6de69e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:07.268831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa36950-c068-4e07-8df6-a22a9d6de69e None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:07.270379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 490/961] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:07.275542 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830891c-22ad-4149-9845-33a35805f77a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:07.275955 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830891c-22ad-4149-9845-33a35805f77a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:07.276145 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3830891c-22ad-4149-9845-33a35805f77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:07.295072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-3830891c-22ad-4149-9845-33a35805f77a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:07.296284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-3830891c-22ad-4149-9845-33a35805f77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:07.366325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 472/962] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:07.630967 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa36950-c068-4e07-8df6-a22a9d6de69e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:07.725261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:07.738105 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa36950-c068-4e07-8df6-a22a9d6de69e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:07.744130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4729b-fb3e-4af6-82c1-80925cbf5742 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:07.744413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4729b-fb3e-4af6-82c1-80925cbf5742 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:07.744687 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4729b-fb3e-4af6-82c1-80925cbf5742 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:07.746852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 473/963] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:07.752037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316f484-1ed4-40bf-a118-66942ea660e5 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:07.752318 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316f484-1ed4-40bf-a118-66942ea660e5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:07.752584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316f484-1ed4-40bf-a118-66942ea660e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:07.771237 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b316f484-1ed4-40bf-a118-66942ea660e5 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:07.772439 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b316f484-1ed4-40bf-a118-66942ea660e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:07.775142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d3a2e-ea02-46ca-9322-0cc6487901ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:07.775936 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d3a2e-ea02-46ca-9322-0cc6487901ab None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:07.776867 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d3a2e-ea02-46ca-9322-0cc6487901ab None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:07.777279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d3a2e-ea02-46ca-9322-0cc6487901ab None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:07.788013 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d3a2e-ea02-46ca-9322-0cc6487901ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-985d3a2e-ea02-46ca-9322-0cc6487901ab None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:35:07.793553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:07.794689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:07.795719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:07.795719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:07.795719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:07.795719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:07.795719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:35:07.795719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:35:07.795719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:35:07.795719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:35:07.795719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:35:07.795719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:35:07.795719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 19:35:07.795719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:07.798055 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 491/964] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Sep 14 19:35:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:07.805971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d3a2e-ea02-46ca-9322-0cc6487901ab None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:07.857046 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 474/965] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:35:07.919272 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d3a2e-ea02-46ca-9322-0cc6487901ab None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:07.957185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc761ce8-6046-48cc-9c10-4ea0d1921cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:07.958143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc761ce8-6046-48cc-9c10-4ea0d1921cb7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:07.958560 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc761ce8-6046-48cc-9c10-4ea0d1921cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:07.958999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc761ce8-6046-48cc-9c10-4ea0d1921cb7 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:07.959806 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc761ce8-6046-48cc-9c10-4ea0d1921cb7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:07.960246 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc761ce8-6046-48cc-9c10-4ea0d1921cb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:07.961740 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc761ce8-6046-48cc-9c10-4ea0d1921cb7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:07.971449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 492/966] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Sep 14 19:35:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:07.977291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316f484-1ed4-40bf-a118-66942ea660e5 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:07.980948 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:08.072234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99881f0e-30cf-40e5-87c0-d3dc5263baef None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:08.072617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99881f0e-30cf-40e5-87c0-d3dc5263baef None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:08.072969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99881f0e-30cf-40e5-87c0-d3dc5263baef None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:08.075193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 493/967] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:35:08.080807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316f484-1ed4-40bf-a118-66942ea660e5 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:08.081136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d733a51-4330-46cd-9b3e-71b73832d7c3 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:08.081488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d733a51-4330-46cd-9b3e-71b73832d7c3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:08.081877 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d733a51-4330-46cd-9b3e-71b73832d7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:08.101273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6d733a51-4330-46cd-9b3e-71b73832d7c3 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:08.103257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6d733a51-4330-46cd-9b3e-71b73832d7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:08.122158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:08.123964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d834f-5e66-4c00-98d2-cdbef57ad494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:08.124753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d834f-5e66-4c00-98d2-cdbef57ad494 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:08.125194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d834f-5e66-4c00-98d2-cdbef57ad494 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:08.125631 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8d834f-5e66-4c00-98d2-cdbef57ad494 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:08.136449 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d834f-5e66-4c00-98d2-cdbef57ad494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:08.138159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8d834f-5e66-4c00-98d2-cdbef57ad494 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-6d8d834f-5e66-4c00-98d2-cdbef57ad494 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:35:08.152963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:08.154018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:35:08.155119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:35:08.155119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:08.155119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:08.157955 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 475/968] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:08.159097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d834f-5e66-4c00-98d2-cdbef57ad494 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:08.181593 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 494/969] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:35:08.186697 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d733a51-4330-46cd-9b3e-71b73832d7c3 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:08.220999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:08.269793 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8d834f-5e66-4c00-98d2-cdbef57ad494 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:08.300966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d733a51-4330-46cd-9b3e-71b73832d7c3 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:08.307071 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50c6a-b563-4f88-80c4-2387371b8c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:08.308692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50c6a-b563-4f88-80c4-2387371b8c89 None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:08.309112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50c6a-b563-4f88-80c4-2387371b8c89 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:08.309632 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e50c6a-b563-4f88-80c4-2387371b8c89 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:08.310524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e50c6a-b563-4f88-80c4-2387371b8c89 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:08.312241 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e50c6a-b563-4f88-80c4-2387371b8c89 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:08.338625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa83238-90d5-422c-81c8-8c211d663de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:08.342165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa83238-90d5-422c-81c8-8c211d663de9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:08.342165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa83238-90d5-422c-81c8-8c211d663de9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:08.342165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa83238-90d5-422c-81c8-8c211d663de9 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:08.342165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa83238-90d5-422c-81c8-8c211d663de9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:08.342165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa83238-90d5-422c-81c8-8c211d663de9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:08.342165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa83238-90d5-422c-81c8-8c211d663de9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:08.350576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:08.354099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 495/970] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:08.359785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa83238-90d5-422c-81c8-8c211d663de9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:08.361868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.sql.core [None req-e8e50c6a-b563-4f88-80c4-2387371b8c89 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 19:35:08.361868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 14 19:35:08.361868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [parameters: {'user_id': 'b042095defba47628dc2289733d71e02', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 14 19:35:08.361868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93426) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-e8e50c6a-b563-4f88-80c4-2387371b8c89 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application conn.query(q) Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application result.read() Sep 14 19:35:08.370829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 19:35:08.377557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application next(self.gen) Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application yield resource Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application next(self.gen) Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise self.value Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application session.commit() Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 19:35:08.379034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.session.flush() Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application compat.raise_( Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 19:35:08.379921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 19:35:08.380837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application conn.query(q) Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application result.read() Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 19:35:08.381689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b042095defba47628dc2289733d71e02', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:08.385126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 14 19:35:08.387232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 14 19:35:08.389534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 14 19:35:08.389534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 14 19:35:08.389534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 19:35:08.389534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 19:35:08.389534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 14 19:35:08.389534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:08.390762 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 476/971] 10.222.0.42 () {60 vars in 1131 bytes} [Wed Sep 14 19:35:08 2022] POST /identity/v3/users => generated 160 bytes in 214 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:08.392894 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50c6a-b563-4f88-80c4-2387371b8c89 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:08.460645 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa83238-90d5-422c-81c8-8c211d663de9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:08.495882 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0b7d3-5c61-4a03-a9d9-59c001d1ce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:08.496543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0b7d3-5c61-4a03-a9d9-59c001d1ce4c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:08.496817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0b7d3-5c61-4a03-a9d9-59c001d1ce4c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:08.497085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0b7d3-5c61-4a03-a9d9-59c001d1ce4c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:08.505537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0b7d3-5c61-4a03-a9d9-59c001d1ce4c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:08.507050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0b7d3-5c61-4a03-a9d9-59c001d1ce4c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:08.509152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e50c6a-b563-4f88-80c4-2387371b8c89 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-57a0b7d3-5c61-4a03-a9d9-59c001d1ce4c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:35:08.519650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:08.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:08.521871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:08.521871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:08.521871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:35:08.521871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:35:08.521871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:35:08.521871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:35:08.521871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:35:08.521871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:35:08.521871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 19:35:08.521871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:08.521871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 496/972] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 19:35:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:08.527282 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0b7d3-5c61-4a03-a9d9-59c001d1ce4c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:08.532645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:08.550728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571b5c5-8224-4f47-a961-3e4f6201c1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:08.551360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571b5c5-8224-4f47-a961-3e4f6201c1b6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:08.551640 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571b5c5-8224-4f47-a961-3e4f6201c1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:08.551902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571b5c5-8224-4f47-a961-3e4f6201c1b6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:08.560546 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571b5c5-8224-4f47-a961-3e4f6201c1b6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:08.562196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4571b5c5-8224-4f47-a961-3e4f6201c1b6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-4571b5c5-8224-4f47-a961-3e4f6201c1b6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:35:08.573670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:08.580185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:08.581242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:08.581242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:08.581242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:35:08.581242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:35:08.581242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:35:08.581242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:35:08.581242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:35:08.581242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:35:08.581242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 19:35:08.581242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:08.581242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 477/973] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:35:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:08.581242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571b5c5-8224-4f47-a961-3e4f6201c1b6 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:08.599498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:08.641332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0b7d3-5c61-4a03-a9d9-59c001d1ce4c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:08.673495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea57948-1667-4f2b-ab6d-7dcd707b799b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:08.674152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea57948-1667-4f2b-ab6d-7dcd707b799b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:08.674453 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea57948-1667-4f2b-ab6d-7dcd707b799b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:08.674716 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea57948-1667-4f2b-ab6d-7dcd707b799b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:08.675281 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea57948-1667-4f2b-ab6d-7dcd707b799b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:08.675963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571b5c5-8224-4f47-a961-3e4f6201c1b6 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:08.676629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea57948-1667-4f2b-ab6d-7dcd707b799b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-fea57948-1667-4f2b-ab6d-7dcd707b799b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:08.685606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:08.686840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:08.687855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 497/974] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 19:35:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:08.688848 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea57948-1667-4f2b-ab6d-7dcd707b799b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:08.712620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd1f2a8-868a-42c2-840f-73f2c65c9b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:08.713257 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1f2a8-868a-42c2-840f-73f2c65c9b56 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:08.713423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1f2a8-868a-42c2-840f-73f2c65c9b56 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:08.713690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd1f2a8-868a-42c2-840f-73f2c65c9b56 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:08.714454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1f2a8-868a-42c2-840f-73f2c65c9b56 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:08.714752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1f2a8-868a-42c2-840f-73f2c65c9b56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:08.716149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd1f2a8-868a-42c2-840f-73f2c65c9b56 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:08.735483 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 478/975] 10.222.0.42 () {56 vars in 1149 bytes} [Wed Sep 14 19:35:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:08.736671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a5bea-d4f4-4bff-b1ad-c7f68f1cd826 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:08.736950 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a5bea-d4f4-4bff-b1ad-c7f68f1cd826 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:08.737218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a5bea-d4f4-4bff-b1ad-c7f68f1cd826 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:08.738553 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 479/976] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:08.739134 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a5bea-d4f4-4bff-b1ad-c7f68f1cd826 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:08.772660 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:08.791380 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea57948-1667-4f2b-ab6d-7dcd707b799b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:08.814475 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:08.825562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408f1bd-39ed-4c0a-95c3-dddf8e24a910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:08.826203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408f1bd-39ed-4c0a-95c3-dddf8e24a910 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:08.826483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408f1bd-39ed-4c0a-95c3-dddf8e24a910 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:08.826770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408f1bd-39ed-4c0a-95c3-dddf8e24a910 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:08.827435 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408f1bd-39ed-4c0a-95c3-dddf8e24a910 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:08.827733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408f1bd-39ed-4c0a-95c3-dddf8e24a910 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:08.829190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408f1bd-39ed-4c0a-95c3-dddf8e24a910 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:08.841149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a5bea-d4f4-4bff-b1ad-c7f68f1cd826 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:08.863689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 498/977] 10.222.0.42 () {56 vars in 1101 bytes} [Wed Sep 14 19:35:08 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:08.864524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408f1bd-39ed-4c0a-95c3-dddf8e24a910 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:08.891474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9866bcb-7734-4669-8579-99e7174e855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:08.892250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9866bcb-7734-4669-8579-99e7174e855b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:08.892628 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9866bcb-7734-4669-8579-99e7174e855b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:08.892997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9866bcb-7734-4669-8579-99e7174e855b None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:08.893745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9866bcb-7734-4669-8579-99e7174e855b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:08.894223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9866bcb-7734-4669-8579-99e7174e855b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:08.895733 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9866bcb-7734-4669-8579-99e7174e855b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:08.901503 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 480/978] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 19:35:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:08.905439 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caec5fd-9478-4237-aeba-2aed16d75622 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:08.905720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caec5fd-9478-4237-aeba-2aed16d75622 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:08.906025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caec5fd-9478-4237-aeba-2aed16d75622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:08.928744 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-2caec5fd-9478-4237-aeba-2aed16d75622 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:08.930529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2caec5fd-9478-4237-aeba-2aed16d75622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:08.989274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408f1bd-39ed-4c0a-95c3-dddf8e24a910 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:09.013042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 481/979] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:09.014136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caec5fd-9478-4237-aeba-2aed16d75622 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:09.032314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39e2bf-72ac-4d8b-a12d-c2ea00124251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:09.033182 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39e2bf-72ac-4d8b-a12d-c2ea00124251 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:09.033599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39e2bf-72ac-4d8b-a12d-c2ea00124251 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:09.034073 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39e2bf-72ac-4d8b-a12d-c2ea00124251 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:09.045109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:09.094162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:09.115495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39e2bf-72ac-4d8b-a12d-c2ea00124251 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:09.116938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b39e2bf-72ac-4d8b-a12d-c2ea00124251 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:09.134700 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caec5fd-9478-4237-aeba-2aed16d75622 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:09.181156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e85f8-ef0e-4890-b25f-246b44611808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:09.182042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e85f8-ef0e-4890-b25f-246b44611808 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:09.182340 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e85f8-ef0e-4890-b25f-246b44611808 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:09.182682 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e85f8-ef0e-4890-b25f-246b44611808 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:09.183295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e85f8-ef0e-4890-b25f-246b44611808 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:09.183596 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e85f8-ef0e-4890-b25f-246b44611808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:09.186454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185e85f8-ef0e-4890-b25f-246b44611808 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:09.194015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 482/980] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 19:35:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:09.202129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e85f8-ef0e-4890-b25f-246b44611808 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:09.237657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 499/981] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:35:08 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:35:09.243344 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39e2bf-72ac-4d8b-a12d-c2ea00124251 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:09.310403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:09.319818 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e85f8-ef0e-4890-b25f-246b44611808 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:09.325307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:09.342018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b39e2bf-72ac-4d8b-a12d-c2ea00124251 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:09.354049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3ecf1-a434-4262-bcec-54441d42e76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:09.354674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3ecf1-a434-4262-bcec-54441d42e76a None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:09.354948 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3ecf1-a434-4262-bcec-54441d42e76a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:09.355213 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e3ecf1-a434-4262-bcec-54441d42e76a None admin] PATH_INFO: `/v3/endpoints` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:09.355941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3ecf1-a434-4262-bcec-54441d42e76a None admin] RBAC: Adding query filter params (interface=admin, service_id=542944ae2e50451f9e3d5201e5f9d834, region_id=RegionOne) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:09.356241 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3ecf1-a434-4262-bcec-54441d42e76a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:09.357581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e3ecf1-a434-4262-bcec-54441d42e76a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:09.362469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 483/982] 10.222.0.42 () {56 vars in 1249 bytes} [Wed Sep 14 19:35:09 2022] GET /identity/v3/endpoints?service_id=542944ae2e50451f9e3d5201e5f9d834&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:09.367287 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3ecf1-a434-4262-bcec-54441d42e76a None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:09.375764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c4e06-a585-4315-9e1a-45a9a758a8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:09.376392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4e06-a585-4315-9e1a-45a9a758a8d0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:09.376668 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4e06-a585-4315-9e1a-45a9a758a8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:09.376936 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c4e06-a585-4315-9e1a-45a9a758a8d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:09.387148 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c4e06-a585-4315-9e1a-45a9a758a8d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:09.388581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3c4e06-a585-4315-9e1a-45a9a758a8d0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-bd3c4e06-a585-4315-9e1a-45a9a758a8d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:35:09.403163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:09.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:35:09.405018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:35:09.405018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:09.405018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:09.405018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 500/983] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:09.405018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c4e06-a585-4315-9e1a-45a9a758a8d0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:09.462862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e3ecf1-a434-4262-bcec-54441d42e76a None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:09.496122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927be04b-01b7-4060-a4bb-d5ebafa8e598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:09.496648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927be04b-01b7-4060-a4bb-d5ebafa8e598 None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:09.496920 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927be04b-01b7-4060-a4bb-d5ebafa8e598 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:09.497297 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927be04b-01b7-4060-a4bb-d5ebafa8e598 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:09.497850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927be04b-01b7-4060-a4bb-d5ebafa8e598 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:09.499195 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927be04b-01b7-4060-a4bb-d5ebafa8e598 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:09.502826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3c4e06-a585-4315-9e1a-45a9a758a8d0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:09.535272 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b673ff-9963-4d91-a12c-d8a4e967b87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:09.535860 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b673ff-9963-4d91-a12c-d8a4e967b87b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:09.536135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b673ff-9963-4d91-a12c-d8a4e967b87b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:09.536407 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b673ff-9963-4d91-a12c-d8a4e967b87b None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:09.536992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b673ff-9963-4d91-a12c-d8a4e967b87b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:09.538357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b673ff-9963-4d91-a12c-d8a4e967b87b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:09.539338 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:09.545143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 501/984] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 19:35:09 2022] GET /identity/v3/services => generated 1497 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:35:09.545921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b673ff-9963-4d91-a12c-d8a4e967b87b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:09.572756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 484/985] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 19:35:09 2022] POST /identity/v3/services => generated 242 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:35:09.666056 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b673ff-9963-4d91-a12c-d8a4e967b87b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:09.713600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03386020-9908-4b85-9922-53e076b8a4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:09.714341 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03386020-9908-4b85-9922-53e076b8a4bf None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:09.714624 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03386020-9908-4b85-9922-53e076b8a4bf None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:09.714958 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03386020-9908-4b85-9922-53e076b8a4bf None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:09.715684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03386020-9908-4b85-9922-53e076b8a4bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:09.716068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03386020-9908-4b85-9922-53e076b8a4bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:09.717481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03386020-9908-4b85-9922-53e076b8a4bf None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:09.737901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 502/986] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:35:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:09.738901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927be04b-01b7-4060-a4bb-d5ebafa8e598 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:09.761292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:09.845421 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927be04b-01b7-4060-a4bb-d5ebafa8e598 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:09.886744 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dca9a4-79a0-4f59-a716-e0d1f9e1d535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:09.887375 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dca9a4-79a0-4f59-a716-e0d1f9e1d535 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:09.887648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dca9a4-79a0-4f59-a716-e0d1f9e1d535 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:09.887920 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dca9a4-79a0-4f59-a716-e0d1f9e1d535 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:09.894653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dca9a4-79a0-4f59-a716-e0d1f9e1d535 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:09.896086 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dca9a4-79a0-4f59-a716-e0d1f9e1d535 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-c5dca9a4-79a0-4f59-a716-e0d1f9e1d535 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:35:09.907726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:09.908694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:09.909653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:35:09.909653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:35:09.909653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:35:09.909653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:35:09.909653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:35:09.909653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:35:09.909653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:35:09.909653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:09.909653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 485/987] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:35:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:09.914833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03386020-9908-4b85-9922-53e076b8a4bf None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:09.973007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:10.018308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03386020-9908-4b85-9922-53e076b8a4bf None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:10.053294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77845f0f-6a6a-43bd-b5f3-eca7e2520779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:10.053933 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77845f0f-6a6a-43bd-b5f3-eca7e2520779 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:10.054213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77845f0f-6a6a-43bd-b5f3-eca7e2520779 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:10.054481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77845f0f-6a6a-43bd-b5f3-eca7e2520779 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:10.055158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77845f0f-6a6a-43bd-b5f3-eca7e2520779 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:10.055458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77845f0f-6a6a-43bd-b5f3-eca7e2520779 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:10.056871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77845f0f-6a6a-43bd-b5f3-eca7e2520779 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:10.074267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 503/988] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:35:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:10.075703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dca9a4-79a0-4f59-a716-e0d1f9e1d535 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:10.154277 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:10.176771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dca9a4-79a0-4f59-a716-e0d1f9e1d535 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:10.211303 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75e896-b686-4727-86b8-2627e8261920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:10.211905 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75e896-b686-4727-86b8-2627e8261920 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:10.212186 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75e896-b686-4727-86b8-2627e8261920 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:10.212456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75e896-b686-4727-86b8-2627e8261920 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:10.213173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75e896-b686-4727-86b8-2627e8261920 None admin] RBAC: Adding query filter params (role.id=688c3a60f8d34392878a70a5abaffefe, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=7d5605f645a74b1b842733dda256193c) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:10.213478 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75e896-b686-4727-86b8-2627e8261920 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:10.215452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75e896-b686-4727-86b8-2627e8261920 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:10.221383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 486/989] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:35:10 2022] GET /identity/v3/role_assignments?user.id=7d5605f645a74b1b842733dda256193c&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=688c3a60f8d34392878a70a5abaffefe => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:11.726192 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d6053-9d5d-477f-9293-e2f357e6273f None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:11.726749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d6053-9d5d-477f-9293-e2f357e6273f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:11.726749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d6053-9d5d-477f-9293-e2f357e6273f None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:11.728099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 504/990] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:11.733325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee6eb1-a80c-4cb9-ba24-e4c40a9e4d46 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:11.733702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee6eb1-a80c-4cb9-ba24-e4c40a9e4d46 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:11.733909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee6eb1-a80c-4cb9-ba24-e4c40a9e4d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:11.752344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-83ee6eb1-a80c-4cb9-ba24-e4c40a9e4d46 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:11.753532 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-83ee6eb1-a80c-4cb9-ba24-e4c40a9e4d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:11.774361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0607-2901-4638-8d40-d8d37c988b1e None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:11.774651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0607-2901-4638-8d40-d8d37c988b1e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:11.774927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa0607-2901-4638-8d40-d8d37c988b1e None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:11.776376 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 505/991] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:11.781590 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830d664-c866-47ea-a6dc-e09ed94e3630 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:11.781900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830d664-c866-47ea-a6dc-e09ed94e3630 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:11.782174 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830d664-c866-47ea-a6dc-e09ed94e3630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:11.800666 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6830d664-c866-47ea-a6dc-e09ed94e3630 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:11.801924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6830d664-c866-47ea-a6dc-e09ed94e3630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:11.823841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 487/992] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:11.881188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 506/993] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:35:12.106132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee6eb1-a80c-4cb9-ba24-e4c40a9e4d46 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:12.151857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830d664-c866-47ea-a6dc-e09ed94e3630 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:12.188964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:12.209618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:12.231840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ee6eb1-a80c-4cb9-ba24-e4c40a9e4d46 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:12.268414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257204b6-b91e-47d1-8773-532b890b78e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:12.269047 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257204b6-b91e-47d1-8773-532b890b78e6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:12.269328 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257204b6-b91e-47d1-8773-532b890b78e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:12.269602 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257204b6-b91e-47d1-8773-532b890b78e6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:12.270210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257204b6-b91e-47d1-8773-532b890b78e6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:12.271572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257204b6-b91e-47d1-8773-532b890b78e6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:12.274777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830d664-c866-47ea-a6dc-e09ed94e3630 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-257204b6-b91e-47d1-8773-532b890b78e6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:12.280942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:12.281953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:12.283116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 488/994] 10.222.0.42 () {56 vars in 1126 bytes} [Wed Sep 14 19:35:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:12.288004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257204b6-b91e-47d1-8773-532b890b78e6 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:12.311786 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de49632-e57a-45da-a026-d341d0dad53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:12.312444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de49632-e57a-45da-a026-d341d0dad53d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:12.312722 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de49632-e57a-45da-a026-d341d0dad53d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:12.312997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de49632-e57a-45da-a026-d341d0dad53d None admin] PATH_INFO: `/v3/services/compute` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:12.313590 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de49632-e57a-45da-a026-d341d0dad53d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:12.315053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de49632-e57a-45da-a026-d341d0dad53d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-2de49632-e57a-45da-a026-d341d0dad53d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:12.325852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:12.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:12.327935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 507/995] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:35:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:12.333093 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de49632-e57a-45da-a026-d341d0dad53d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:12.404125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257204b6-b91e-47d1-8773-532b890b78e6 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:12.437492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafc1d3-e8ae-422f-bc8e-1bd818410bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:12.438125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafc1d3-e8ae-422f-bc8e-1bd818410bf4 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:12.438401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafc1d3-e8ae-422f-bc8e-1bd818410bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:12.438669 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafc1d3-e8ae-422f-bc8e-1bd818410bf4 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:12.439285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cafc1d3-e8ae-422f-bc8e-1bd818410bf4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:12.439583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cafc1d3-e8ae-422f-bc8e-1bd818410bf4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:12.440591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de49632-e57a-45da-a026-d341d0dad53d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:12.440936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cafc1d3-e8ae-422f-bc8e-1bd818410bf4 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:12.442004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:12.446411 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 489/996] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 19:35:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:12.447539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bea76-6887-4a84-bcb0-e3b8ee48a4e5 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:12.447821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bea76-6887-4a84-bcb0-e3b8ee48a4e5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:12.448193 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bea76-6887-4a84-bcb0-e3b8ee48a4e5 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:12.449922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 490/997] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:12.451929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:12.454565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bea76-6887-4a84-bcb0-e3b8ee48a4e5 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:12.475255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51106b4d-6976-414a-af84-ee86eb5d992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:12.475852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51106b4d-6976-414a-af84-ee86eb5d992b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:12.476476 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51106b4d-6976-414a-af84-ee86eb5d992b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:12.476476 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51106b4d-6976-414a-af84-ee86eb5d992b None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:12.476968 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51106b4d-6976-414a-af84-ee86eb5d992b None admin] RBAC: Adding query filter params (name=compute) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:12.477266 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51106b4d-6976-414a-af84-ee86eb5d992b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:12.478642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51106b4d-6976-414a-af84-ee86eb5d992b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:12.483490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 508/998] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 19:35:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:12.484621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fda50-32d3-473c-97d6-67a9953771c9 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:12.484903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fda50-32d3-473c-97d6-67a9953771c9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:12.485173 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fda50-32d3-473c-97d6-67a9953771c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:12.503205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-177fda50-32d3-473c-97d6-67a9953771c9 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:12.504406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-177fda50-32d3-473c-97d6-67a9953771c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:12.555639 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bea76-6887-4a84-bcb0-e3b8ee48a4e5 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:12.570423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 509/999] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:12.571243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fda50-32d3-473c-97d6-67a9953771c9 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:12.589106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a88ab-35bf-4fd7-82c9-8b2b2f56d360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:12.589712 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a88ab-35bf-4fd7-82c9-8b2b2f56d360 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:12.590069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a88ab-35bf-4fd7-82c9-8b2b2f56d360 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:12.590338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a88ab-35bf-4fd7-82c9-8b2b2f56d360 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:12.590947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a88ab-35bf-4fd7-82c9-8b2b2f56d360 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:12.591241 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a88ab-35bf-4fd7-82c9-8b2b2f56d360 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:12.592581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a88ab-35bf-4fd7-82c9-8b2b2f56d360 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:12.597300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 491/1000] 10.222.0.42 () {56 vars in 1122 bytes} [Wed Sep 14 19:35:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:12.602897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a88ab-35bf-4fd7-82c9-8b2b2f56d360 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:12.668019 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:12.682101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fda50-32d3-473c-97d6-67a9953771c9 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:12.684154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:12.707776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a88ab-35bf-4fd7-82c9-8b2b2f56d360 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:12.720610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43516c-edbb-47db-bbaa-bd174e7891ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:12.721220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43516c-edbb-47db-bbaa-bd174e7891ed None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:12.721492 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43516c-edbb-47db-bbaa-bd174e7891ed None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:12.721758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c43516c-edbb-47db-bbaa-bd174e7891ed None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:12.722396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43516c-edbb-47db-bbaa-bd174e7891ed None admin] RBAC: Adding query filter params (type=compute) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:12.722692 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43516c-edbb-47db-bbaa-bd174e7891ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:12.724040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c43516c-edbb-47db-bbaa-bd174e7891ed None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:12.736882 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 510/1001] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 19:35:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:12.737449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43516c-edbb-47db-bbaa-bd174e7891ed None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:12.752671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434860c-a572-4ea1-9109-19e5502c6b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:12.753266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434860c-a572-4ea1-9109-19e5502c6b3c None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:12.753535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434860c-a572-4ea1-9109-19e5502c6b3c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:12.753823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4434860c-a572-4ea1-9109-19e5502c6b3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:12.754469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434860c-a572-4ea1-9109-19e5502c6b3c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:12.756049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4434860c-a572-4ea1-9109-19e5502c6b3c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:12.827127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 492/1002] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:35:12 2022] POST /identity/v3/endpoints => generated 325 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:35:12.834219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c43516c-edbb-47db-bbaa-bd174e7891ed None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:12.843357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434860c-a572-4ea1-9109-19e5502c6b3c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:12.885205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5df84e-3edf-4a9f-bca8-21a70e990f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:12.885961 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5df84e-3edf-4a9f-bca8-21a70e990f73 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:12.886301 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5df84e-3edf-4a9f-bca8-21a70e990f73 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:12.886684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5df84e-3edf-4a9f-bca8-21a70e990f73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:12.887401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5df84e-3edf-4a9f-bca8-21a70e990f73 None admin] RBAC: Adding query filter params (interface=public, service_id=9a239af68aef4a36a6bad21f82dda7de, region_id=RegionOne) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:12.887818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5df84e-3edf-4a9f-bca8-21a70e990f73 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:12.889284 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5df84e-3edf-4a9f-bca8-21a70e990f73 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:12.896867 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 511/1003] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 19:35:12 2022] GET /identity/v3/endpoints?service_id=9a239af68aef4a36a6bad21f82dda7de&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:12.902665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5df84e-3edf-4a9f-bca8-21a70e990f73 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:12.933155 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:12.964498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4434860c-a572-4ea1-9109-19e5502c6b3c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:12.990706 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:13.005618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cecc45-a471-4046-806d-a5df7a68f268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:13.006284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cecc45-a471-4046-806d-a5df7a68f268 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:13.006667 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cecc45-a471-4046-806d-a5df7a68f268 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:13.006937 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cecc45-a471-4046-806d-a5df7a68f268 None admin] PATH_INFO: `/v3/services/image` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:13.008332 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cecc45-a471-4046-806d-a5df7a68f268 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:13.008872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cecc45-a471-4046-806d-a5df7a68f268 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:13.012496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5df84e-3edf-4a9f-bca8-21a70e990f73 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-40cecc45-a471-4046-806d-a5df7a68f268 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:13.020243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:13.021357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:13.023225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 493/1004] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Sep 14 19:35:12 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:13.028602 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cecc45-a471-4046-806d-a5df7a68f268 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:13.055949 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9f249-0f64-4c9a-b119-e140534b477c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:13.056566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9f249-0f64-4c9a-b119-e140534b477c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:13.056836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9f249-0f64-4c9a-b119-e140534b477c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:13.057113 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a9f249-0f64-4c9a-b119-e140534b477c None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:13.058312 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9f249-0f64-4c9a-b119-e140534b477c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:13.059671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a9f249-0f64-4c9a-b119-e140534b477c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:13.066393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 512/1005] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 19:35:12 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:35:13.145036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cecc45-a471-4046-806d-a5df7a68f268 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:13.182537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd772c-a209-4419-894c-22990e844a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:13.183583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd772c-a209-4419-894c-22990e844a67 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:13.184067 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd772c-a209-4419-894c-22990e844a67 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:13.184546 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd772c-a209-4419-894c-22990e844a67 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:13.185411 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd772c-a209-4419-894c-22990e844a67 None admin] RBAC: Adding query filter params (name=image) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:13.185939 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd772c-a209-4419-894c-22990e844a67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:13.187531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd772c-a209-4419-894c-22990e844a67 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:13.198168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 494/1006] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:13.199465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9f249-0f64-4c9a-b119-e140534b477c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:13.294280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:13.303538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a9f249-0f64-4c9a-b119-e140534b477c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:13.339589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8597c-1133-4a87-b5d9-1752dadee6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:13.339589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8597c-1133-4a87-b5d9-1752dadee6b9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:13.339589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8597c-1133-4a87-b5d9-1752dadee6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:13.339589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8597c-1133-4a87-b5d9-1752dadee6b9 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:13.340160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8597c-1133-4a87-b5d9-1752dadee6b9 None admin] RBAC: Adding query filter params (type=image) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:13.340406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8597c-1133-4a87-b5d9-1752dadee6b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:13.341856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8597c-1133-4a87-b5d9-1752dadee6b9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:13.346740 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 513/1007] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:13 2022] GET /identity/v3/services?type=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:15.019521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a65db9-6356-4cf4-a7d7-e98c4d688c1d None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:15.020052 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a65db9-6356-4cf4-a7d7-e98c4d688c1d None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:15.020052 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a65db9-6356-4cf4-a7d7-e98c4d688c1d None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:15.021531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 495/1008] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:15.027015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bd902-bbad-4a5a-aad4-938f4f4ac3fc None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:15.027466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bd902-bbad-4a5a-aad4-938f4f4ac3fc None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:15.027638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bd902-bbad-4a5a-aad4-938f4f4ac3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:15.046512 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5a2bd902-bbad-4a5a-aad4-938f4f4ac3fc None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:15.047767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5a2bd902-bbad-4a5a-aad4-938f4f4ac3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:15.118021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 514/1009] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:15.215719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3ce88-357b-4cc4-8f22-abcd0b5d6d22 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:15.216012 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3ce88-357b-4cc4-8f22-abcd0b5d6d22 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:15.216173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e3ce88-357b-4cc4-8f22-abcd0b5d6d22 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:15.217630 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 496/1010] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:15.222647 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f9df4-6c66-4623-9ad9-847b023bc471 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:15.222931 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f9df4-6c66-4623-9ad9-847b023bc471 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:15.223202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866f9df4-6c66-4623-9ad9-847b023bc471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:15.241563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-866f9df4-6c66-4623-9ad9-847b023bc471 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:15.242763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-866f9df4-6c66-4623-9ad9-847b023bc471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:15.307684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 515/1011] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:15.387299 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3ce88-357b-4cc4-8f22-abcd0b5d6d22 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:15.439565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:15.489904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e3ce88-357b-4cc4-8f22-abcd0b5d6d22 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:15.493273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750bdbf-6c7e-4a66-89d9-dc26055674ec None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:15.493559 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750bdbf-6c7e-4a66-89d9-dc26055674ec None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:15.493891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0750bdbf-6c7e-4a66-89d9-dc26055674ec None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:15.495851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 516/1012] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:15.500587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71fbf6-eeaa-4673-b54c-79044cb4611e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:15.500866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71fbf6-eeaa-4673-b54c-79044cb4611e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:15.501134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a71fbf6-eeaa-4673-b54c-79044cb4611e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:15.519548 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1a71fbf6-eeaa-4673-b54c-79044cb4611e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:15.520767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1a71fbf6-eeaa-4673-b54c-79044cb4611e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:15.526762 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb87535-d211-4161-be30-db330510ae40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:15.527485 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb87535-d211-4161-be30-db330510ae40 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:15.527692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb87535-d211-4161-be30-db330510ae40 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:15.528036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb87535-d211-4161-be30-db330510ae40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:15.529074 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb87535-d211-4161-be30-db330510ae40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:15.530425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:15.530538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb87535-d211-4161-be30-db330510ae40 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:15.532234 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 497/1013] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:15.537939 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb87535-d211-4161-be30-db330510ae40 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:15.591157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 517/1014] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:35:15.591991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71fbf6-eeaa-4673-b54c-79044cb4611e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:15.644447 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb87535-d211-4161-be30-db330510ae40 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:15.662076 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:15.679184 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1271a-49d3-4d0e-ae7b-10bdd2a67826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:15.679788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1271a-49d3-4d0e-ae7b-10bdd2a67826 None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:15.680064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1271a-49d3-4d0e-ae7b-10bdd2a67826 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:15.680341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b1271a-49d3-4d0e-ae7b-10bdd2a67826 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:15.680974 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1271a-49d3-4d0e-ae7b-10bdd2a67826 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:15.682690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b1271a-49d3-4d0e-ae7b-10bdd2a67826 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:15.695449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a71fbf6-eeaa-4673-b54c-79044cb4611e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:15.729194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509b8a0-dcd4-45c4-b4a2-c095bcb9398e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:15.729843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509b8a0-dcd4-45c4-b4a2-c095bcb9398e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:15.730124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509b8a0-dcd4-45c4-b4a2-c095bcb9398e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:15.730391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8509b8a0-dcd4-45c4-b4a2-c095bcb9398e None admin] PATH_INFO: `/v3/services/compute` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:15.731574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509b8a0-dcd4-45c4-b4a2-c095bcb9398e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:15.732980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8509b8a0-dcd4-45c4-b4a2-c095bcb9398e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-8509b8a0-dcd4-45c4-b4a2-c095bcb9398e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:15.741586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:15.742813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:15.743662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 518/1015] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:35:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:15.748943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509b8a0-dcd4-45c4-b4a2-c095bcb9398e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:15.759574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:15.842808 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8509b8a0-dcd4-45c4-b4a2-c095bcb9398e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:15.853266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 498/1016] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 19:35:15 2022] POST /identity/v3/projects => generated 311 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:35:15.861873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1271a-49d3-4d0e-ae7b-10bdd2a67826 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:15.886975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632e6af-747f-4b34-9a26-61daa970f57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:15.887600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632e6af-747f-4b34-9a26-61daa970f57e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:15.887870 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632e6af-747f-4b34-9a26-61daa970f57e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:15.888137 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6632e6af-747f-4b34-9a26-61daa970f57e None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:15.888792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632e6af-747f-4b34-9a26-61daa970f57e None admin] RBAC: Adding query filter params (name=compute) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:15.889092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632e6af-747f-4b34-9a26-61daa970f57e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:15.890517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6632e6af-747f-4b34-9a26-61daa970f57e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:15.900813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 519/1017] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 19:35:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:15.902202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632e6af-747f-4b34-9a26-61daa970f57e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:15.987053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:15.987259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b1271a-49d3-4d0e-ae7b-10bdd2a67826 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:16.017481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6632e6af-747f-4b34-9a26-61daa970f57e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:16.024185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3866e-1a91-403a-aad9-c77bbc2c288d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:16.024884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3866e-1a91-403a-aad9-c77bbc2c288d None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:16.025574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3866e-1a91-403a-aad9-c77bbc2c288d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:16.025574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3866e-1a91-403a-aad9-c77bbc2c288d None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:16.026623 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3866e-1a91-403a-aad9-c77bbc2c288d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:16.027991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3866e-1a91-403a-aad9-c77bbc2c288d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:16.053483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d75ed-c652-41ca-91ef-dfee1204384a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:16.054133 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d75ed-c652-41ca-91ef-dfee1204384a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:16.054409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d75ed-c652-41ca-91ef-dfee1204384a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:16.054676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8d75ed-c652-41ca-91ef-dfee1204384a None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:16.055292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d75ed-c652-41ca-91ef-dfee1204384a None admin] RBAC: Adding query filter params (type=compute) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:16.055593 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d75ed-c652-41ca-91ef-dfee1204384a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:16.056979 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8d75ed-c652-41ca-91ef-dfee1204384a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:16.061813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 520/1018] 10.222.0.42 () {56 vars in 1112 bytes} [Wed Sep 14 19:35:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:16.067798 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d75ed-c652-41ca-91ef-dfee1204384a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:16.167438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8d75ed-c652-41ca-91ef-dfee1204384a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:16.180664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 499/1019] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 19:35:15 2022] POST /identity/v3/services => generated 242 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:35:16.206633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:16.213112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a681f8e-e67e-4c2b-b7b2-bed513875b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:16.214290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a681f8e-e67e-4c2b-b7b2-bed513875b9c None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:16.214631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a681f8e-e67e-4c2b-b7b2-bed513875b9c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:16.214912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a681f8e-e67e-4c2b-b7b2-bed513875b9c None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:16.215678 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a681f8e-e67e-4c2b-b7b2-bed513875b9c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:16.217256 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a681f8e-e67e-4c2b-b7b2-bed513875b9c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:16.312535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 521/1020] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:35:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:35:18.060038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebfa03-46a0-4363-b4b1-d3956a6753af None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:18.060557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebfa03-46a0-4363-b4b1-d3956a6753af None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:18.060557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebfa03-46a0-4363-b4b1-d3956a6753af None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:18.061967 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 500/1021] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:18.067366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955de719-cb66-4a19-8eb3-daac2a3737c0 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:18.067760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955de719-cb66-4a19-8eb3-daac2a3737c0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:18.067953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955de719-cb66-4a19-8eb3-daac2a3737c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:18.086569 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-955de719-cb66-4a19-8eb3-daac2a3737c0 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:18.087763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-955de719-cb66-4a19-8eb3-daac2a3737c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:18.162468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 522/1022] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:18.327504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d692f79-04d0-4c7c-b979-f56678b3789f None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:18.327849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d692f79-04d0-4c7c-b979-f56678b3789f None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:18.328044 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d692f79-04d0-4c7c-b979-f56678b3789f None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:18.329520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 501/1023] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:18.334566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd1c64-0377-45e0-bcd9-e3270fc8500d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:18.334852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd1c64-0377-45e0-bcd9-e3270fc8500d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:18.335127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dd1c64-0377-45e0-bcd9-e3270fc8500d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:18.354270 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a3dd1c64-0377-45e0-bcd9-e3270fc8500d None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:18.355451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a3dd1c64-0377-45e0-bcd9-e3270fc8500d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:18.420607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 523/1024] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:18.445764 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d692f79-04d0-4c7c-b979-f56678b3789f None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:18.482285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:18.554588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d692f79-04d0-4c7c-b979-f56678b3789f None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:18.565672 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0017e78-df0d-4108-a019-c8d73392ea98 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:18.565985 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0017e78-df0d-4108-a019-c8d73392ea98 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:18.566244 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0017e78-df0d-4108-a019-c8d73392ea98 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:18.567870 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 524/1025] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:35:18.573649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5d5df-8d92-413c-a723-03526caa22c2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:18.573970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5d5df-8d92-413c-a723-03526caa22c2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:18.574239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d5d5df-8d92-413c-a723-03526caa22c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:18.594514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-79d5d5df-8d92-413c-a723-03526caa22c2 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:18.595937 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-79d5d5df-8d92-413c-a723-03526caa22c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:18.596507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b316b-660a-4b20-8e98-62367fbdda21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:18.597138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b316b-660a-4b20-8e98-62367fbdda21 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:18.597417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b316b-660a-4b20-8e98-62367fbdda21 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:18.597690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b316b-660a-4b20-8e98-62367fbdda21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:18.598883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b316b-660a-4b20-8e98-62367fbdda21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:18.600309 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b316b-660a-4b20-8e98-62367fbdda21 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:18.601968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 502/1026] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:18.607644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b316b-660a-4b20-8e98-62367fbdda21 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:18.643332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:18.673046 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 525/1027] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:35:18.689192 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5d5df-8d92-413c-a723-03526caa22c2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:18.719900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:18.723945 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b316b-660a-4b20-8e98-62367fbdda21 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:18.757110 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cc3e4-d594-4f23-85a6-6a9f2f75598e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:18.757706 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cc3e4-d594-4f23-85a6-6a9f2f75598e None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:18.758021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cc3e4-d594-4f23-85a6-6a9f2f75598e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:18.758376 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994cc3e4-d594-4f23-85a6-6a9f2f75598e None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:18.758952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cc3e4-d594-4f23-85a6-6a9f2f75598e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:18.760911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994cc3e4-d594-4f23-85a6-6a9f2f75598e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:18.793707 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d5d5df-8d92-413c-a723-03526caa22c2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:18.827027 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13032669-1be7-45fb-9dd1-ce44b040fcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:18.827652 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13032669-1be7-45fb-9dd1-ce44b040fcaa None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:18.827926 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13032669-1be7-45fb-9dd1-ce44b040fcaa None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:18.828197 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13032669-1be7-45fb-9dd1-ce44b040fcaa None admin] PATH_INFO: `/v3/services/image` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:18.828766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13032669-1be7-45fb-9dd1-ce44b040fcaa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:18.830214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13032669-1be7-45fb-9dd1-ce44b040fcaa None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-13032669-1be7-45fb-9dd1-ce44b040fcaa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:18.839459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:18.840508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:18.841423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 526/1028] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Sep 14 19:35:18 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:18.846406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13032669-1be7-45fb-9dd1-ce44b040fcaa None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:18.877707 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 503/1029] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:35:18 2022] POST /identity/v3/users => generated 285 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:35:18.893432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:18.947693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cc3e4-d594-4f23-85a6-6a9f2f75598e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:18.966838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13032669-1be7-45fb-9dd1-ce44b040fcaa None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:19.007780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbc435-4d1b-4588-8c3f-c8440e27b5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:19.008415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbc435-4d1b-4588-8c3f-c8440e27b5b3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:19.008710 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbc435-4d1b-4588-8c3f-c8440e27b5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:19.008971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbc435-4d1b-4588-8c3f-c8440e27b5b3 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:19.009618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbc435-4d1b-4588-8c3f-c8440e27b5b3 None admin] RBAC: Adding query filter params (name=image) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:19.009969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbc435-4d1b-4588-8c3f-c8440e27b5b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:19.011504 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbc435-4d1b-4588-8c3f-c8440e27b5b3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:19.017228 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 527/1030] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:19.023276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbc435-4d1b-4588-8c3f-c8440e27b5b3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:19.081773 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994cc3e4-d594-4f23-85a6-6a9f2f75598e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:19.121481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87381f05-69d9-40d3-a155-4858b4d4f71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:19.122152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87381f05-69d9-40d3-a155-4858b4d4f71c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:19.122429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87381f05-69d9-40d3-a155-4858b4d4f71c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:19.122698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87381f05-69d9-40d3-a155-4858b4d4f71c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:19.133847 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbc435-4d1b-4588-8c3f-c8440e27b5b3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:19.134912 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87381f05-69d9-40d3-a155-4858b4d4f71c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:35:19.138195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-87381f05-69d9-40d3-a155-4858b4d4f71c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:35:19.140348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:19.141286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:19.142183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:19.142183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:19.142183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:19.142183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:19.142183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:35:19.142183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:35:19.142183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:35:19.142183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:35:19.142183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:35:19.142183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:35:19.142183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 19:35:19.142183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:19.145879 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 504/1031] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Sep 14 19:35:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:19.147698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87381f05-69d9-40d3-a155-4858b4d4f71c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:19.163841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:19.170036 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1e800-2209-4c47-8bb9-7afa8974c29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:19.170646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1e800-2209-4c47-8bb9-7afa8974c29c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:19.170966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1e800-2209-4c47-8bb9-7afa8974c29c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:19.171243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed1e800-2209-4c47-8bb9-7afa8974c29c None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:19.171917 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1e800-2209-4c47-8bb9-7afa8974c29c None admin] RBAC: Adding query filter params (type=image) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:19.172555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1e800-2209-4c47-8bb9-7afa8974c29c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:19.174004 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed1e800-2209-4c47-8bb9-7afa8974c29c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:19.179415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 528/1032] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:19.185904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1e800-2209-4c47-8bb9-7afa8974c29c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:19.254577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87381f05-69d9-40d3-a155-4858b4d4f71c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:19.288310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1f4af-42e6-4bb8-98b8-304c6685c81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:19.289089 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1f4af-42e6-4bb8-98b8-304c6685c81b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:19.289369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1f4af-42e6-4bb8-98b8-304c6685c81b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:19.289640 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1f4af-42e6-4bb8-98b8-304c6685c81b None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:19.290402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1f4af-42e6-4bb8-98b8-304c6685c81b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:19.290710 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1f4af-42e6-4bb8-98b8-304c6685c81b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:19.292118 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1f4af-42e6-4bb8-98b8-304c6685c81b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:19.294147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed1e800-2209-4c47-8bb9-7afa8974c29c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:19.300159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 505/1033] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Sep 14 19:35:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:19.305835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1f4af-42e6-4bb8-98b8-304c6685c81b None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:19.338664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b75570-ed79-45d3-92a7-b05ba0ed7d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:19.339292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b75570-ed79-45d3-92a7-b05ba0ed7d56 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:19.339572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b75570-ed79-45d3-92a7-b05ba0ed7d56 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:19.339846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b75570-ed79-45d3-92a7-b05ba0ed7d56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:19.340529 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b75570-ed79-45d3-92a7-b05ba0ed7d56 None admin] RBAC: Adding query filter params (interface=public, service_id=a805b725c76044a1bcad226e266f8412, region_id=RegionOne) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:19.340837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b75570-ed79-45d3-92a7-b05ba0ed7d56 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:19.342293 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b75570-ed79-45d3-92a7-b05ba0ed7d56 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:19.347285 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 529/1034] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 19:35:19 2022] GET /identity/v3/endpoints?service_id=a805b725c76044a1bcad226e266f8412&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:19.352845 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b75570-ed79-45d3-92a7-b05ba0ed7d56 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:19.386695 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:19.392836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:19.417577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1f4af-42e6-4bb8-98b8-304c6685c81b None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:19.451754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690ff09-f70f-44c0-852a-f880fcfeac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:19.453026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690ff09-f70f-44c0-852a-f880fcfeac9a None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:19.453723 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690ff09-f70f-44c0-852a-f880fcfeac9a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:19.454698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690ff09-f70f-44c0-852a-f880fcfeac9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:19.460540 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b75570-ed79-45d3-92a7-b05ba0ed7d56 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:19.465272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690ff09-f70f-44c0-852a-f880fcfeac9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:19.466748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690ff09-f70f-44c0-852a-f880fcfeac9a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-a690ff09-f70f-44c0-852a-f880fcfeac9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:35:19.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:19.483321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:35:19.484232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:35:19.484232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:19.484232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:19.486173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 506/1035] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:19.489298 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690ff09-f70f-44c0-852a-f880fcfeac9a None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:19.498789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a819a9-7c1f-4c14-beda-d5c2821f7078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:19.499826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a819a9-7c1f-4c14-beda-d5c2821f7078 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:19.500110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a819a9-7c1f-4c14-beda-d5c2821f7078 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:19.500381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a819a9-7c1f-4c14-beda-d5c2821f7078 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:19.501041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a819a9-7c1f-4c14-beda-d5c2821f7078 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:19.503122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a819a9-7c1f-4c14-beda-d5c2821f7078 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:19.510276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 530/1036] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 19:35:19 2022] GET /identity/v3/services => generated 1957 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:35:19.601045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690ff09-f70f-44c0-852a-f880fcfeac9a None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:19.628933 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:19.638541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232ea21-fb50-41de-bbf3-3ca1ab433545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:19.639181 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232ea21-fb50-41de-bbf3-3ca1ab433545 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:19.639457 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232ea21-fb50-41de-bbf3-3ca1ab433545 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:19.639833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b232ea21-fb50-41de-bbf3-3ca1ab433545 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:19.640488 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232ea21-fb50-41de-bbf3-3ca1ab433545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:19.640790 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232ea21-fb50-41de-bbf3-3ca1ab433545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:19.642207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b232ea21-fb50-41de-bbf3-3ca1ab433545 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:19.660545 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 507/1037] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:19.662938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a819a9-7c1f-4c14-beda-d5c2821f7078 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:19.758947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:19.766079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a819a9-7c1f-4c14-beda-d5c2821f7078 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:19.797999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ddbab-36a2-4117-ac08-1b64aaf30f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:19.798635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ddbab-36a2-4117-ac08-1b64aaf30f37 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:19.798911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ddbab-36a2-4117-ac08-1b64aaf30f37 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:19.799182 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ddbab-36a2-4117-ac08-1b64aaf30f37 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:19.807956 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ddbab-36a2-4117-ac08-1b64aaf30f37 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:19.809420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ddbab-36a2-4117-ac08-1b64aaf30f37 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-cc1ddbab-36a2-4117-ac08-1b64aaf30f37 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:35:19.822143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:19.823758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:19.824720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:19.824720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:19.824720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:35:19.824720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:35:19.824720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:35:19.824720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:35:19.824720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:35:19.824720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:35:19.824720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:35:19.824720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:19.824720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 531/1038] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 19:35:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:19.829875 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232ea21-fb50-41de-bbf3-3ca1ab433545 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:19.935176 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b232ea21-fb50-41de-bbf3-3ca1ab433545 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:19.969207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883df67-1233-4e7e-8e95-9ce27cd34dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:19.969822 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883df67-1233-4e7e-8e95-9ce27cd34dec None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:19.970115 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883df67-1233-4e7e-8e95-9ce27cd34dec None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:19.970387 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883df67-1233-4e7e-8e95-9ce27cd34dec None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:19.971145 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883df67-1233-4e7e-8e95-9ce27cd34dec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:19.971455 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883df67-1233-4e7e-8e95-9ce27cd34dec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:19.972879 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883df67-1233-4e7e-8e95-9ce27cd34dec None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:19.990434 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 508/1039] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 19:35:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:19.996158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ddbab-36a2-4117-ac08-1b64aaf30f37 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:20.098397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ddbab-36a2-4117-ac08-1b64aaf30f37 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:20.130929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db780600-756e-4c38-9890-16caf1112f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:20.131533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db780600-756e-4c38-9890-16caf1112f75 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:20.131814 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db780600-756e-4c38-9890-16caf1112f75 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:20.132084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db780600-756e-4c38-9890-16caf1112f75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:20.156819 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:20.200100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db780600-756e-4c38-9890-16caf1112f75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:20.201552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db780600-756e-4c38-9890-16caf1112f75 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:20.300212 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 532/1040] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:35:19 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 19:35:20.306115 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883df67-1233-4e7e-8e95-9ce27cd34dec None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:20.352435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:20.407063 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883df67-1233-4e7e-8e95-9ce27cd34dec None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:20.441344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfaeba-704b-46c2-9709-c97a211fe17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:20.442023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfaeba-704b-46c2-9709-c97a211fe17e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:20.442570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfaeba-704b-46c2-9709-c97a211fe17e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:20.442843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfaeba-704b-46c2-9709-c97a211fe17e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:20.452685 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfaeba-704b-46c2-9709-c97a211fe17e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:20.454149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfaeba-704b-46c2-9709-c97a211fe17e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-8ddfaeba-704b-46c2-9709-c97a211fe17e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:35:20.468761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:20.469715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:35:20.470910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:35:20.470910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:20.470910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:20.473889 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 509/1041] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:20.475903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db780600-756e-4c38-9890-16caf1112f75 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:20.530244 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:20.574761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db780600-756e-4c38-9890-16caf1112f75 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:20.607732 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2398c6d-5cc9-4cad-a2e7-00c67a360a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:20.608269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2398c6d-5cc9-4cad-a2e7-00c67a360a81 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:20.608546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2398c6d-5cc9-4cad-a2e7-00c67a360a81 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:20.608819 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2398c6d-5cc9-4cad-a2e7-00c67a360a81 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:20.609452 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2398c6d-5cc9-4cad-a2e7-00c67a360a81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:20.609754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2398c6d-5cc9-4cad-a2e7-00c67a360a81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:20.611137 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2398c6d-5cc9-4cad-a2e7-00c67a360a81 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:20.622994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 533/1042] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:20.628611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfaeba-704b-46c2-9709-c97a211fe17e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:20.727073 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:20.731476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfaeba-704b-46c2-9709-c97a211fe17e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:20.764778 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefd015-12d7-4b8f-af33-d739a69e48c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:20.765495 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefd015-12d7-4b8f-af33-d739a69e48c3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:20.765771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefd015-12d7-4b8f-af33-d739a69e48c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:20.766060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fefd015-12d7-4b8f-af33-d739a69e48c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:20.772786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fefd015-12d7-4b8f-af33-d739a69e48c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:20.774542 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fefd015-12d7-4b8f-af33-d739a69e48c3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-8fefd015-12d7-4b8f-af33-d739a69e48c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:35:20.787049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:20.787935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:20.788850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:35:20.788850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:35:20.788850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:35:20.788850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:35:20.788850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:35:20.788850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:35:20.788850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:35:20.788850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:20.789877 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 510/1043] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:35:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:20.795058 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2398c6d-5cc9-4cad-a2e7-00c67a360a81 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:20.897638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2398c6d-5cc9-4cad-a2e7-00c67a360a81 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:20.912519 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:20.931359 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee61e6fb-7077-47e0-9291-8bc9ae296759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:20.931994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61e6fb-7077-47e0-9291-8bc9ae296759 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:20.932355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61e6fb-7077-47e0-9291-8bc9ae296759 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:20.932628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee61e6fb-7077-47e0-9291-8bc9ae296759 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:20.933295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61e6fb-7077-47e0-9291-8bc9ae296759 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:20.933593 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61e6fb-7077-47e0-9291-8bc9ae296759 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:20.935037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee61e6fb-7077-47e0-9291-8bc9ae296759 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:20.953173 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 534/1044] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:35:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:20.956755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefd015-12d7-4b8f-af33-d739a69e48c3 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:21.060788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fefd015-12d7-4b8f-af33-d739a69e48c3 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:21.079662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5af4a-bb71-4d9f-8bfd-2259b0142611 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:21.079983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5af4a-bb71-4d9f-8bfd-2259b0142611 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:21.080206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5af4a-bb71-4d9f-8bfd-2259b0142611 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:21.081667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 535/1045] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:35:21.086387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6758ad-fcaf-4093-b97c-3e70ccdc8c7c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:21.086670 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6758ad-fcaf-4093-b97c-3e70ccdc8c7c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:21.086945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6758ad-fcaf-4093-b97c-3e70ccdc8c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:21.094556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68565070-c76f-4ed5-9f0f-8778fb182f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:21.095154 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68565070-c76f-4ed5-9f0f-8778fb182f7a None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:21.095427 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68565070-c76f-4ed5-9f0f-8778fb182f7a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:21.095692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68565070-c76f-4ed5-9f0f-8778fb182f7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:21.096393 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68565070-c76f-4ed5-9f0f-8778fb182f7a None admin] RBAC: Adding query filter params (role.id=688c3a60f8d34392878a70a5abaffefe, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=2a4284b59f804dc59674d621d87634bc) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:21.096692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68565070-c76f-4ed5-9f0f-8778fb182f7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:21.098120 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68565070-c76f-4ed5-9f0f-8778fb182f7a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:21.108407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 511/1046] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:35:20 2022] GET /identity/v3/role_assignments?user.id=2a4284b59f804dc59674d621d87634bc&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=688c3a60f8d34392878a70a5abaffefe => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:21.108690 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3a6758ad-fcaf-4093-b97c-3e70ccdc8c7c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:21.108690 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3a6758ad-fcaf-4093-b97c-3e70ccdc8c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:21.186144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 536/1047] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:35:21.450444 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68565070-c76f-4ed5-9f0f-8778fb182f7a None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:21.475029 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:21.549377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68565070-c76f-4ed5-9f0f-8778fb182f7a None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:21.586869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4480fa7-249c-4292-90c1-27289fa9848c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:21.587525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4480fa7-249c-4292-90c1-27289fa9848c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:21.587799 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4480fa7-249c-4292-90c1-27289fa9848c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:21.588065 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4480fa7-249c-4292-90c1-27289fa9848c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:21.600972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4480fa7-249c-4292-90c1-27289fa9848c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-f4480fa7-249c-4292-90c1-27289fa9848c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:35:21.605810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:21.606863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:21.607779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:21.607779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:21.607779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:21.607779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:21.607779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:35:21.607779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:35:21.607779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:35:21.607779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:35:21.607779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:35:21.607779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:35:21.607779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:35:21.607779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:21.610153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 512/1048] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:35:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:21.612854 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6758ad-fcaf-4093-b97c-3e70ccdc8c7c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:21.694024 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6379a40-f099-4373-b341-0b2cf77a69e1 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:21.694309 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6379a40-f099-4373-b341-0b2cf77a69e1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:21.694738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6379a40-f099-4373-b341-0b2cf77a69e1 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:21.696275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 513/1049] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:21.701917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33236a30-5f76-4397-a37a-83c43e3d7abe None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:21.702402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33236a30-5f76-4397-a37a-83c43e3d7abe None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:21.702872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33236a30-5f76-4397-a37a-83c43e3d7abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:21.712185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6758ad-fcaf-4093-b97c-3e70ccdc8c7c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:21.715094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:21.722887 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-33236a30-5f76-4397-a37a-83c43e3d7abe None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:21.725881 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-33236a30-5f76-4397-a37a-83c43e3d7abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:21.749213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522374ee-b834-4440-894a-697fa86f55d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:21.749852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522374ee-b834-4440-894a-697fa86f55d8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:21.750134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522374ee-b834-4440-894a-697fa86f55d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:21.750403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522374ee-b834-4440-894a-697fa86f55d8 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:21.751052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522374ee-b834-4440-894a-697fa86f55d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:21.751344 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522374ee-b834-4440-894a-697fa86f55d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:21.752693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522374ee-b834-4440-894a-697fa86f55d8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:21.762028 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 537/1050] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:35:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:21.767522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522374ee-b834-4440-894a-697fa86f55d8 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:21.796737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 514/1051] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:35:21.874463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522374ee-b834-4440-894a-697fa86f55d8 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:21.909430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91631e1-8392-41d8-986e-fc531e375bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:21.910149 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91631e1-8392-41d8-986e-fc531e375bf0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:21.910431 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91631e1-8392-41d8-986e-fc531e375bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:21.910702 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91631e1-8392-41d8-986e-fc531e375bf0 None admin] PATH_INFO: `/v3/users/157afd8d4fea4805a11a42e0c2bcff18` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:21.924403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91631e1-8392-41d8-986e-fc531e375bf0 None admin] RBAC: Authorizing `identity:get_user(user_id=157afd8d4fea4805a11a42e0c2bcff18)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:21.925943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91631e1-8392-41d8-986e-fc531e375bf0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:21.932590 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 538/1052] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:35:21 2022] GET /identity/v3/users/157afd8d4fea4805a11a42e0c2bcff18 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:21.933976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33236a30-5f76-4397-a37a-83c43e3d7abe None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:21.948980 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:22.036033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33236a30-5f76-4397-a37a-83c43e3d7abe None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:22.066442 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91631e1-8392-41d8-986e-fc531e375bf0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:22.069897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413dabd-0697-4432-b5f2-b57af5cf24a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:22.070505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413dabd-0697-4432-b5f2-b57af5cf24a1 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:22.070779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413dabd-0697-4432-b5f2-b57af5cf24a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:22.071047 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3413dabd-0697-4432-b5f2-b57af5cf24a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:22.093334 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:22.138231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413dabd-0697-4432-b5f2-b57af5cf24a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:22.139671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3413dabd-0697-4432-b5f2-b57af5cf24a1 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:22.153382 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:22.170345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91631e1-8392-41d8-986e-fc531e375bf0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:22.205741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daefad6-e06d-457e-8045-cb41261342b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:22.206408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daefad6-e06d-457e-8045-cb41261342b9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:22.206661 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daefad6-e06d-457e-8045-cb41261342b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:22.206932 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daefad6-e06d-457e-8045-cb41261342b9 None admin] PATH_INFO: `/v3/services/image` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:22.207551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daefad6-e06d-457e-8045-cb41261342b9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:22.208847 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daefad6-e06d-457e-8045-cb41261342b9 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-6daefad6-e06d-457e-8045-cb41261342b9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:22.218156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:22.219239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:22.220096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:22.220096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:22.220096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:22.220096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:22.220096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:22.220096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:22.220096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:22.220096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:22.220096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:22.220096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:22.220096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:35:22.220096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:35:22.220096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 19:35:22.220096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:22.224224 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 539/1053] 10.222.0.42 () {56 vars in 1105 bytes} [Wed Sep 14 19:35:22 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:22.224865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daefad6-e06d-457e-8045-cb41261342b9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:22.241110 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 515/1054] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:35:21 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 19:35:22.247330 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413dabd-0697-4432-b5f2-b57af5cf24a1 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:22.322760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:22.337060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daefad6-e06d-457e-8045-cb41261342b9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:22.356166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3413dabd-0697-4432-b5f2-b57af5cf24a1 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:22.369639 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:22.378229 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa5e05-67e6-4bb0-94f4-33eb74dc0abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:22.382291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa5e05-67e6-4bb0-94f4-33eb74dc0abd None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:22.382291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa5e05-67e6-4bb0-94f4-33eb74dc0abd None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:22.382291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa5e05-67e6-4bb0-94f4-33eb74dc0abd None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:22.382291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa5e05-67e6-4bb0-94f4-33eb74dc0abd None admin] RBAC: Adding query filter params (name=image) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:22.382291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa5e05-67e6-4bb0-94f4-33eb74dc0abd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:22.382291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faa5e05-67e6-4bb0-94f4-33eb74dc0abd None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:22.391539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 540/1055] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:22.392210 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa5e05-67e6-4bb0-94f4-33eb74dc0abd None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:22.393089 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee48b6c-82d6-4c23-a875-1c8df9cdab75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:22.393720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee48b6c-82d6-4c23-a875-1c8df9cdab75 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:22.394057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee48b6c-82d6-4c23-a875-1c8df9cdab75 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:22.394322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee48b6c-82d6-4c23-a875-1c8df9cdab75 None admin] PATH_INFO: `/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:22.395517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee48b6c-82d6-4c23-a875-1c8df9cdab75 None admin] RBAC: Authorizing `identity:get_project(project_id=0f7144ff96fd42378ab24f2bce4c6d78)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:22.397762 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee48b6c-82d6-4c23-a875-1c8df9cdab75 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:22.404397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 516/1056] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:35:22 2022] GET /identity/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:22.405868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee48b6c-82d6-4c23-a875-1c8df9cdab75 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:22.494300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa5e05-67e6-4bb0-94f4-33eb74dc0abd None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:22.507574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee48b6c-82d6-4c23-a875-1c8df9cdab75 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:22.534766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf04fd-1f96-411e-9b73-7eb09cc9a7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:22.535428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf04fd-1f96-411e-9b73-7eb09cc9a7d0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:22.535702 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf04fd-1f96-411e-9b73-7eb09cc9a7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:22.535970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf04fd-1f96-411e-9b73-7eb09cc9a7d0 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:22.536639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf04fd-1f96-411e-9b73-7eb09cc9a7d0 None admin] RBAC: Adding query filter params (type=image) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:22.536932 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf04fd-1f96-411e-9b73-7eb09cc9a7d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:22.538310 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf04fd-1f96-411e-9b73-7eb09cc9a7d0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:22.543435 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 541/1057] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:22.548698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79189b64-9e47-4f0a-a018-06086a2b448c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:22.548952 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf04fd-1f96-411e-9b73-7eb09cc9a7d0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:22.549459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79189b64-9e47-4f0a-a018-06086a2b448c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:22.549824 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79189b64-9e47-4f0a-a018-06086a2b448c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:22.550131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79189b64-9e47-4f0a-a018-06086a2b448c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:22.550847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79189b64-9e47-4f0a-a018-06086a2b448c None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=0f7144ff96fd42378ab24f2bce4c6d78, user.id=157afd8d4fea4805a11a42e0c2bcff18) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:22.551155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79189b64-9e47-4f0a-a018-06086a2b448c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:22.553602 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79189b64-9e47-4f0a-a018-06086a2b448c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:22.569825 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 517/1058] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:35:22 2022] GET /identity/v3/role_assignments?user.id=157afd8d4fea4805a11a42e0c2bcff18&scope.project.id=0f7144ff96fd42378ab24f2bce4c6d78&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:22.570058 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:22.657484 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf04fd-1f96-411e-9b73-7eb09cc9a7d0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:22.694119 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78628d81-8c57-47fd-85b7-aab22bb66a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:22.694741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78628d81-8c57-47fd-85b7-aab22bb66a8e None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:22.695017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78628d81-8c57-47fd-85b7-aab22bb66a8e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:22.695287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78628d81-8c57-47fd-85b7-aab22bb66a8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:22.695970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78628d81-8c57-47fd-85b7-aab22bb66a8e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:22.697358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78628d81-8c57-47fd-85b7-aab22bb66a8e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:22.803361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 542/1059] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:35:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:35:23.276527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20605177-e233-45ec-a84a-784371e7d68c None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:23.276922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20605177-e233-45ec-a84a-784371e7d68c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:23.277126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20605177-e233-45ec-a84a-784371e7d68c None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:23.278496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 518/1060] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:23.283570 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387e34d-e269-4bdd-930e-1840c8f30a3e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:23.284029 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387e34d-e269-4bdd-930e-1840c8f30a3e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:23.284120 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b387e34d-e269-4bdd-930e-1840c8f30a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:23.302916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b387e34d-e269-4bdd-930e-1840c8f30a3e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:23.304115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b387e34d-e269-4bdd-930e-1840c8f30a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:23.374697 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:23.381837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 543/1061] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:23.651415 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20605177-e233-45ec-a84a-784371e7d68c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:23.700957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:23.757381 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20605177-e233-45ec-a84a-784371e7d68c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:23.791996 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a71c0-cbbc-452f-b054-258e5a60001b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:23.792624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a71c0-cbbc-452f-b054-258e5a60001b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:23.792897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a71c0-cbbc-452f-b054-258e5a60001b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:23.793160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a71c0-cbbc-452f-b054-258e5a60001b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:23.803160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a71c0-cbbc-452f-b054-258e5a60001b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-6c6a71c0-cbbc-452f-b054-258e5a60001b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:35:23.807910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:23.808840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:23.809703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:23.809703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:23.809703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:23.809703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:23.809703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:35:23.809703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:35:23.809703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:35:23.809703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:35:23.809703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:35:23.809703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:35:23.809703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 19:35:23.809703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:23.809703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 519/1062] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Sep 14 19:35:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:23.815805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387e34d-e269-4bdd-930e-1840c8f30a3e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:23.930057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b387e34d-e269-4bdd-930e-1840c8f30a3e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:23.973313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482a745-9e2a-41ea-a880-eebfc4ffcd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:23.973959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482a745-9e2a-41ea-a880-eebfc4ffcd70 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:23.974240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482a745-9e2a-41ea-a880-eebfc4ffcd70 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:23.974512 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482a745-9e2a-41ea-a880-eebfc4ffcd70 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:23.975165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482a745-9e2a-41ea-a880-eebfc4ffcd70 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:23.975468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482a745-9e2a-41ea-a880-eebfc4ffcd70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:23.976826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482a745-9e2a-41ea-a880-eebfc4ffcd70 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:23.989396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 544/1063] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Sep 14 19:35:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:23.990936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a71c0-cbbc-452f-b054-258e5a60001b None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:24.092740 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a71c0-cbbc-452f-b054-258e5a60001b None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:24.095171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:24.127392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71870d-8f22-4677-b880-0f05541fdeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:24.128010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71870d-8f22-4677-b880-0f05541fdeb6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:24.128283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71870d-8f22-4677-b880-0f05541fdeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:24.128545 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71870d-8f22-4677-b880-0f05541fdeb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:24.138680 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71870d-8f22-4677-b880-0f05541fdeb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:24.140428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a71870d-8f22-4677-b880-0f05541fdeb6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-4a71870d-8f22-4677-b880-0f05541fdeb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:35:24.154925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:24.155900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:35:24.156800 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:35:24.156800 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:24.156800 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:24.156800 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 520/1064] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:24.165831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482a745-9e2a-41ea-a880-eebfc4ffcd70 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:24.226867 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:24.267485 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482a745-9e2a-41ea-a880-eebfc4ffcd70 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:24.299931 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327ec8f-25ba-4a31-b406-915d08d3dffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:24.300555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327ec8f-25ba-4a31-b406-915d08d3dffe None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:24.300798 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327ec8f-25ba-4a31-b406-915d08d3dffe None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:24.301067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327ec8f-25ba-4a31-b406-915d08d3dffe None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:24.301701 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327ec8f-25ba-4a31-b406-915d08d3dffe None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:24.302029 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327ec8f-25ba-4a31-b406-915d08d3dffe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:24.303380 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327ec8f-25ba-4a31-b406-915d08d3dffe None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:24.321238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 545/1065] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:24.323225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71870d-8f22-4677-b880-0f05541fdeb6 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:24.421471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a71870d-8f22-4677-b880-0f05541fdeb6 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:24.455593 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89967d9-5bb3-4757-9aa6-c04be084e42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:24.456217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89967d9-5bb3-4757-9aa6-c04be084e42a None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:24.456493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89967d9-5bb3-4757-9aa6-c04be084e42a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:24.456763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89967d9-5bb3-4757-9aa6-c04be084e42a None admin] PATH_INFO: `/v3/users/nova` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:24.464842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89967d9-5bb3-4757-9aa6-c04be084e42a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:24.466315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89967d9-5bb3-4757-9aa6-c04be084e42a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-a89967d9-5bb3-4757-9aa6-c04be084e42a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:35:24.477271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:24.478436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:24.479464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:24.479464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:24.479464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:35:24.479464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:35:24.479464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:35:24.479464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:35:24.479464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:35:24.479464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:35:24.479464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:35:24.479464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:24.479464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 521/1066] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 19:35:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:24.484144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327ec8f-25ba-4a31-b406-915d08d3dffe None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:24.590333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327ec8f-25ba-4a31-b406-915d08d3dffe None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:24.612836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:24.627706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c623a8-1fe7-4766-a6cf-5f98b96b0196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:24.628321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c623a8-1fe7-4766-a6cf-5f98b96b0196 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:24.628597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c623a8-1fe7-4766-a6cf-5f98b96b0196 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:24.628865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c623a8-1fe7-4766-a6cf-5f98b96b0196 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:24.629629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c623a8-1fe7-4766-a6cf-5f98b96b0196 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:24.629973 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c623a8-1fe7-4766-a6cf-5f98b96b0196 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:24.631381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c623a8-1fe7-4766-a6cf-5f98b96b0196 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:24.653891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 546/1067] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 19:35:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:24.656018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89967d9-5bb3-4757-9aa6-c04be084e42a None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:24.675089 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:24.692268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ebeaf-e218-4ccf-afa4-14018c34469b None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:24.692558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ebeaf-e218-4ccf-afa4-14018c34469b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:24.693205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ebeaf-e218-4ccf-afa4-14018c34469b None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:24.694280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 547/1068] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:35:24.699633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840192e8-aebd-4c92-bb51-2cbcd893728a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:24.699913 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840192e8-aebd-4c92-bb51-2cbcd893728a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:24.700180 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840192e8-aebd-4c92-bb51-2cbcd893728a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:24.719473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-840192e8-aebd-4c92-bb51-2cbcd893728a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:24.720663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-840192e8-aebd-4c92-bb51-2cbcd893728a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:24.763785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89967d9-5bb3-4757-9aa6-c04be084e42a None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:24.790474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 548/1069] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:35:24.799579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdcb5a-b62a-4fe1-b6cf-c27ad9821814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:24.800265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdcb5a-b62a-4fe1-b6cf-c27ad9821814 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:24.800542 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdcb5a-b62a-4fe1-b6cf-c27ad9821814 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:24.800808 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdcb5a-b62a-4fe1-b6cf-c27ad9821814 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:24.811781 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdcb5a-b62a-4fe1-b6cf-c27ad9821814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:24.813273 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdcb5a-b62a-4fe1-b6cf-c27ad9821814 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-5ccdcb5a-b62a-4fe1-b6cf-c27ad9821814 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:35:24.828061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:24.829009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:35:24.830030 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:35:24.830030 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:24.830030 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:24.830030 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 522/1070] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:24.834829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840192e8-aebd-4c92-bb51-2cbcd893728a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:24.943660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840192e8-aebd-4c92-bb51-2cbcd893728a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:24.952807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:24.976985 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0a7fe-d6e5-4c72-8e03-22602169e209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:24.977587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0a7fe-d6e5-4c72-8e03-22602169e209 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:24.977891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0a7fe-d6e5-4c72-8e03-22602169e209 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:24.978176 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c0a7fe-d6e5-4c72-8e03-22602169e209 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:24.978774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0a7fe-d6e5-4c72-8e03-22602169e209 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:24.979616 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0a7fe-d6e5-4c72-8e03-22602169e209 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:24.980963 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c0a7fe-d6e5-4c72-8e03-22602169e209 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:24.989472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d90467-13ca-47f0-8878-fa520a7afa74 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:24.990080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d90467-13ca-47f0-8878-fa520a7afa74 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:24.990425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d90467-13ca-47f0-8878-fa520a7afa74 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:24.991966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 523/1071] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:35:24.997836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 549/1072] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:24.997977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fa0b1-44db-40b8-a65e-9aae016f0362 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:24.997977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fa0b1-44db-40b8-a65e-9aae016f0362 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:24.997977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fa0b1-44db-40b8-a65e-9aae016f0362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:25.000036 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0a7fe-d6e5-4c72-8e03-22602169e209 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:25.016577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-035fa0b1-44db-40b8-a65e-9aae016f0362 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:25.017673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-035fa0b1-44db-40b8-a65e-9aae016f0362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:25.079121 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:25.084718 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 524/1073] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:35:25.085504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fa0b1-44db-40b8-a65e-9aae016f0362 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:25.105723 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c0a7fe-d6e5-4c72-8e03-22602169e209 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:25.140365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488036f-8d7a-4d78-9133-939d428e3038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:25.140989 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488036f-8d7a-4d78-9133-939d428e3038 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:25.141268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488036f-8d7a-4d78-9133-939d428e3038 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:25.141543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488036f-8d7a-4d78-9133-939d428e3038 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:25.148192 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488036f-8d7a-4d78-9133-939d428e3038 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:25.149623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3488036f-8d7a-4d78-9133-939d428e3038 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-3488036f-8d7a-4d78-9133-939d428e3038 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:35:25.161336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:25.162486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:25.163763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:35:25.163763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:35:25.163763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:35:25.163763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:35:25.163763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:35:25.163763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:35:25.163763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:35:25.163763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:25.163763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 550/1074] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:35:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:25.168573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488036f-8d7a-4d78-9133-939d428e3038 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:25.186366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fa0b1-44db-40b8-a65e-9aae016f0362 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:25.197238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:25.220985 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778de3c-e82c-4455-a0e2-a69d040023e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:25.221607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778de3c-e82c-4455-a0e2-a69d040023e3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:25.221909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778de3c-e82c-4455-a0e2-a69d040023e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:25.222186 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778de3c-e82c-4455-a0e2-a69d040023e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:25.232527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778de3c-e82c-4455-a0e2-a69d040023e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-4778de3c-e82c-4455-a0e2-a69d040023e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:35:25.237349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:25.238336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:25.239218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:25.239218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:25.239218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:25.239218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:25.239218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:35:25.239218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:35:25.239218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:35:25.239218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:35:25.239218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:35:25.239218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:35:25.239218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:35:25.239218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:25.239218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 525/1075] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:35:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:25.245002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778de3c-e82c-4455-a0e2-a69d040023e3 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:25.274569 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488036f-8d7a-4d78-9133-939d428e3038 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:25.301180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:25.311211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fff68f-8156-4eec-9249-63700d7b532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:25.311813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fff68f-8156-4eec-9249-63700d7b532c None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:25.315165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fff68f-8156-4eec-9249-63700d7b532c None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:25.315165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fff68f-8156-4eec-9249-63700d7b532c None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:25.315165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fff68f-8156-4eec-9249-63700d7b532c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:25.315165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fff68f-8156-4eec-9249-63700d7b532c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:25.315698 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fff68f-8156-4eec-9249-63700d7b532c None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:25.333961 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 551/1076] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:35:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:25.335944 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fff68f-8156-4eec-9249-63700d7b532c None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:25.345941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778de3c-e82c-4455-a0e2-a69d040023e3 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:25.380904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e36f0f-2a7b-4abe-b6be-acfb9ff74a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:25.381505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e36f0f-2a7b-4abe-b6be-acfb9ff74a12 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:25.381790 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e36f0f-2a7b-4abe-b6be-acfb9ff74a12 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:25.382089 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e36f0f-2a7b-4abe-b6be-acfb9ff74a12 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:25.382743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e36f0f-2a7b-4abe-b6be-acfb9ff74a12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:25.383045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e36f0f-2a7b-4abe-b6be-acfb9ff74a12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:25.384415 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e36f0f-2a7b-4abe-b6be-acfb9ff74a12 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:25.392308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 526/1077] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:35:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:25.393084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e36f0f-2a7b-4abe-b6be-acfb9ff74a12 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:25.428413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:25.440367 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fff68f-8156-4eec-9249-63700d7b532c None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:25.473538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d458de4-04ae-47cd-aa17-76eae8ab1988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:25.474243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d458de4-04ae-47cd-aa17-76eae8ab1988 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:25.474520 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d458de4-04ae-47cd-aa17-76eae8ab1988 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:25.474791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d458de4-04ae-47cd-aa17-76eae8ab1988 None admin] PATH_INFO: `/v3/projects/6bfc6c1c443849038af35577daf34088/users/2a4284b59f804dc59674d621d87634bc/roles/688c3a60f8d34392878a70a5abaffefe` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:25.476947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d458de4-04ae-47cd-aa17-76eae8ab1988 None admin] RBAC: Authorizing `identity:create_grant(project_id=6bfc6c1c443849038af35577daf34088, user_id=2a4284b59f804dc59674d621d87634bc, role_id=688c3a60f8d34392878a70a5abaffefe)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:25.478408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d458de4-04ae-47cd-aa17-76eae8ab1988 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:25.490654 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e36f0f-2a7b-4abe-b6be-acfb9ff74a12 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:25.509692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:25.522819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495b25e-8e01-4345-92e4-a1dfd43a237e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:25.523407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495b25e-8e01-4345-92e4-a1dfd43a237e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:25.523676 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495b25e-8e01-4345-92e4-a1dfd43a237e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:25.523940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495b25e-8e01-4345-92e4-a1dfd43a237e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:25.555912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 552/1078] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:35:25 2022] PUT /identity/v3/projects/6bfc6c1c443849038af35577daf34088/users/2a4284b59f804dc59674d621d87634bc/roles/688c3a60f8d34392878a70a5abaffefe => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:35:25.557768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d458de4-04ae-47cd-aa17-76eae8ab1988 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:25.596667 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495b25e-8e01-4345-92e4-a1dfd43a237e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:25.598177 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495b25e-8e01-4345-92e4-a1dfd43a237e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:25.672974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d458de4-04ae-47cd-aa17-76eae8ab1988 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:25.711240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf490aa-0090-4e3f-8c2f-86306b54fbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:25.711890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf490aa-0090-4e3f-8c2f-86306b54fbe7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:25.712164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf490aa-0090-4e3f-8c2f-86306b54fbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:25.712432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf490aa-0090-4e3f-8c2f-86306b54fbe7 None admin] PATH_INFO: `/v3/users/157afd8d4fea4805a11a42e0c2bcff18` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:25.714145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf490aa-0090-4e3f-8c2f-86306b54fbe7 None admin] RBAC: Authorizing `identity:get_user(user_id=157afd8d4fea4805a11a42e0c2bcff18)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:25.715603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf490aa-0090-4e3f-8c2f-86306b54fbe7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:25.717233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 553/1079] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:35:25 2022] GET /identity/v3/users/157afd8d4fea4805a11a42e0c2bcff18 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:25.718664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 527/1080] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:35:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:35:25.723094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf490aa-0090-4e3f-8c2f-86306b54fbe7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:25.724455 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495b25e-8e01-4345-92e4-a1dfd43a237e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:25.745937 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:25.749206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:25.827214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf490aa-0090-4e3f-8c2f-86306b54fbe7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:25.833185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495b25e-8e01-4345-92e4-a1dfd43a237e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:25.872108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f39188-08c5-4f30-bc3d-d625664516a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:25.873072 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f39188-08c5-4f30-bc3d-d625664516a3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:25.873355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f39188-08c5-4f30-bc3d-d625664516a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:25.873630 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f39188-08c5-4f30-bc3d-d625664516a3 None admin] PATH_INFO: `/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:25.875557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f39188-08c5-4f30-bc3d-d625664516a3 None admin] RBAC: Authorizing `identity:get_project(project_id=0f7144ff96fd42378ab24f2bce4c6d78)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:25.877345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f39188-08c5-4f30-bc3d-d625664516a3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:25.878857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960911a5-5aff-4b10-84e2-c8cc2076b911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:25.884249 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 554/1081] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:35:25 2022] GET /identity/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:25.884378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960911a5-5aff-4b10-84e2-c8cc2076b911 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:25.884378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960911a5-5aff-4b10-84e2-c8cc2076b911 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:25.884378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960911a5-5aff-4b10-84e2-c8cc2076b911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:25.885052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f39188-08c5-4f30-bc3d-d625664516a3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:25.892856 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960911a5-5aff-4b10-84e2-c8cc2076b911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:25.894337 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960911a5-5aff-4b10-84e2-c8cc2076b911 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-960911a5-5aff-4b10-84e2-c8cc2076b911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:35:25.909438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:25.910515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:35:25.911403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:35:25.911403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:25.911403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:25.914104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 528/1082] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:25.917037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960911a5-5aff-4b10-84e2-c8cc2076b911 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:25.977202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:25.991993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:25.994542 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f39188-08c5-4f30-bc3d-d625664516a3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:26.021512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960911a5-5aff-4b10-84e2-c8cc2076b911 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:26.031019 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43ff62-2b33-413e-9f69-797449202084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:26.031793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43ff62-2b33-413e-9f69-797449202084 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:26.032074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43ff62-2b33-413e-9f69-797449202084 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:26.032345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43ff62-2b33-413e-9f69-797449202084 None admin] PATH_INFO: `/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78/users/157afd8d4fea4805a11a42e0c2bcff18/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:26.034317 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43ff62-2b33-413e-9f69-797449202084 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f7144ff96fd42378ab24f2bce4c6d78, user_id=157afd8d4fea4805a11a42e0c2bcff18, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:26.035892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43ff62-2b33-413e-9f69-797449202084 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:26.056294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905e6ee-6fb5-43b8-9720-2f123125bf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:26.057424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905e6ee-6fb5-43b8-9720-2f123125bf3c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:26.057705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905e6ee-6fb5-43b8-9720-2f123125bf3c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:26.058056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905e6ee-6fb5-43b8-9720-2f123125bf3c None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:26.058699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905e6ee-6fb5-43b8-9720-2f123125bf3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:26.058999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905e6ee-6fb5-43b8-9720-2f123125bf3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:26.060475 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905e6ee-6fb5-43b8-9720-2f123125bf3c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:26.073162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 529/1083] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:26.123977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 555/1084] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:35:25 2022] PUT /identity/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78/users/157afd8d4fea4805a11a42e0c2bcff18/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:35:27.690612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856997-369f-4be1-93c6-db030b6bac06 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:27.691173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856997-369f-4be1-93c6-db030b6bac06 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:27.691173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73856997-369f-4be1-93c6-db030b6bac06 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:27.692717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 530/1085] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:27.698371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2778b8-6644-4a58-8b71-285f3b06b2a7 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:27.698808 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2778b8-6644-4a58-8b71-285f3b06b2a7 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:27.698987 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2778b8-6644-4a58-8b71-285f3b06b2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:27.717280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1c2778b8-6644-4a58-8b71-285f3b06b2a7 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:27.718504 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1c2778b8-6644-4a58-8b71-285f3b06b2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:27.783235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 556/1086] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:28.049324 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73856997-369f-4be1-93c6-db030b6bac06 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:28.148209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73856997-369f-4be1-93c6-db030b6bac06 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:28.180210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:28.185140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81def2f3-ac89-4ac1-9253-e03959a782c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:28.185764 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81def2f3-ac89-4ac1-9253-e03959a782c6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:28.186089 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81def2f3-ac89-4ac1-9253-e03959a782c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:28.186354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81def2f3-ac89-4ac1-9253-e03959a782c6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:28.196338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81def2f3-ac89-4ac1-9253-e03959a782c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-81def2f3-ac89-4ac1-9253-e03959a782c6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:35:28.201138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:28.202292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:28.203452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:28.203452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:28.203452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:28.203452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:28.203452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:35:28.203452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:35:28.203452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:35:28.203452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:35:28.203452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:35:28.203452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:35:28.203452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 19:35:28.203452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:28.205962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 531/1087] 10.222.0.42 () {56 vars in 1120 bytes} [Wed Sep 14 19:35:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:28.208125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2778b8-6644-4a58-8b71-285f3b06b2a7 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:28.259335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cb6b3-1dd4-4d13-a2d0-b710822f9b12 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:28.259617 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cb6b3-1dd4-4d13-a2d0-b710822f9b12 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:28.259909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911cb6b3-1dd4-4d13-a2d0-b710822f9b12 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:28.261753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 532/1088] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:35:28.266861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ec8ac-c09e-4e9e-9ef9-71f14951acf4 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:28.267142 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ec8ac-c09e-4e9e-9ef9-71f14951acf4 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:28.267411 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ec8ac-c09e-4e9e-9ef9-71f14951acf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:28.281453 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:28.284626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-1d7ec8ac-c09e-4e9e-9ef9-71f14951acf4 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:28.285852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1d7ec8ac-c09e-4e9e-9ef9-71f14951acf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:28.304927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2778b8-6644-4a58-8b71-285f3b06b2a7 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:28.338270 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1adbca-35ec-497b-847f-b657b55e3fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:28.338873 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1adbca-35ec-497b-847f-b657b55e3fea None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:28.339150 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1adbca-35ec-497b-847f-b657b55e3fea None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:28.339422 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1adbca-35ec-497b-847f-b657b55e3fea None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:28.340074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1adbca-35ec-497b-847f-b657b55e3fea None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:28.340376 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1adbca-35ec-497b-847f-b657b55e3fea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:28.341724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1adbca-35ec-497b-847f-b657b55e3fea None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:28.350187 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 557/1089] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Sep 14 19:35:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:28.351022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 533/1090] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:35:28.356596 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1adbca-35ec-497b-847f-b657b55e3fea None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:28.451436 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1adbca-35ec-497b-847f-b657b55e3fea None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:28.483172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb308e1-e7a0-467b-bbf4-fc2574dda2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:28.483784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb308e1-e7a0-467b-bbf4-fc2574dda2bf None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:28.484055 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb308e1-e7a0-467b-bbf4-fc2574dda2bf None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:28.484320 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb308e1-e7a0-467b-bbf4-fc2574dda2bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:28.486885 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:28.495266 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb308e1-e7a0-467b-bbf4-fc2574dda2bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:28.497380 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb308e1-e7a0-467b-bbf4-fc2574dda2bf None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-4eb308e1-e7a0-467b-bbf4-fc2574dda2bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:35:28.510140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:28.511124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:35:28.512022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:35:28.512022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:28.512022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:28.512022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 558/1091] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:28.516873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ec8ac-c09e-4e9e-9ef9-71f14951acf4 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:28.612462 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ec8ac-c09e-4e9e-9ef9-71f14951acf4 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:28.617184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb308e1-e7a0-467b-bbf4-fc2574dda2bf None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:28.636308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:28.647174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ae103-0480-4da2-9d0f-529bb66e6b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:28.647786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ae103-0480-4da2-9d0f-529bb66e6b36 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:28.648062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ae103-0480-4da2-9d0f-529bb66e6b36 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:28.648379 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ae103-0480-4da2-9d0f-529bb66e6b36 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:28.648992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ae103-0480-4da2-9d0f-529bb66e6b36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:28.649295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ae103-0480-4da2-9d0f-529bb66e6b36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:28.651315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ae103-0480-4da2-9d0f-529bb66e6b36 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:28.662558 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 534/1092] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:28.668093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ae103-0480-4da2-9d0f-529bb66e6b36 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:28.718154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb308e1-e7a0-467b-bbf4-fc2574dda2bf None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:28.750884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce2022-2f1f-4483-8034-8817d1482f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:28.751503 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce2022-2f1f-4483-8034-8817d1482f65 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:28.751776 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce2022-2f1f-4483-8034-8817d1482f65 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:28.752041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce2022-2f1f-4483-8034-8817d1482f65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:28.763038 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce2022-2f1f-4483-8034-8817d1482f65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:35:28.766028 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ae103-0480-4da2-9d0f-529bb66e6b36 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-50ce2022-2f1f-4483-8034-8817d1482f65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:35:28.768042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:28.769112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:28.770250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:28.770250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:28.770250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:28.770250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:28.770250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:35:28.770250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:35:28.770250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:35:28.770250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:35:28.770250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:35:28.770250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:35:28.770250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:35:28.770250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:28.770250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 559/1093] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:35:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:28.775167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce2022-2f1f-4483-8034-8817d1482f65 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:28.803912 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ceda24-2226-4098-942b-b805907f2e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:28.804536 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ceda24-2226-4098-942b-b805907f2e41 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:28.804811 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ceda24-2226-4098-942b-b805907f2e41 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:28.805088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ceda24-2226-4098-942b-b805907f2e41 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:28.812481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ceda24-2226-4098-942b-b805907f2e41 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:28.813930 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ceda24-2226-4098-942b-b805907f2e41 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:28.822194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-a7ceda24-2226-4098-942b-b805907f2e41 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:35:28.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:28.826125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:28.827101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:28.827101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:28.827101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:35:28.827101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:35:28.827101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:35:28.827101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:35:28.827101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:35:28.827101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:35:28.827101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 19:35:28.827101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:28.827101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 535/1094] 10.222.0.42 () {56 vars in 1093 bytes} [Wed Sep 14 19:35:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:28.832025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ceda24-2226-4098-942b-b805907f2e41 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:28.873650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce2022-2f1f-4483-8034-8817d1482f65 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:28.875281 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:28.907141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4d49e-fbef-4f07-b6b9-8f77aac0c817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:28.907746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4d49e-fbef-4f07-b6b9-8f77aac0c817 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:28.908022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4d49e-fbef-4f07-b6b9-8f77aac0c817 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:28.908291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b4d49e-fbef-4f07-b6b9-8f77aac0c817 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:28.908937 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4d49e-fbef-4f07-b6b9-8f77aac0c817 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:28.909236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4d49e-fbef-4f07-b6b9-8f77aac0c817 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:28.910625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b4d49e-fbef-4f07-b6b9-8f77aac0c817 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:28.919016 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 560/1095] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:35:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:28.924900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4d49e-fbef-4f07-b6b9-8f77aac0c817 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:28.931299 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ceda24-2226-4098-942b-b805907f2e41 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:28.964296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21e989-e78e-4157-9714-0e6f1e30d2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:28.964892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21e989-e78e-4157-9714-0e6f1e30d2d8 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:28.965165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21e989-e78e-4157-9714-0e6f1e30d2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:28.965431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21e989-e78e-4157-9714-0e6f1e30d2d8 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:28.966179 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21e989-e78e-4157-9714-0e6f1e30d2d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:28.966477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21e989-e78e-4157-9714-0e6f1e30d2d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:28.967887 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21e989-e78e-4157-9714-0e6f1e30d2d8 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:28.984593 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 536/1096] 10.222.0.42 () {56 vars in 1133 bytes} [Wed Sep 14 19:35:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:28.990354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21e989-e78e-4157-9714-0e6f1e30d2d8 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:29.024340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b4d49e-fbef-4f07-b6b9-8f77aac0c817 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:29.034834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:29.061016 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1efd00-6998-409e-9117-79ffeaa7d8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:29.061658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1efd00-6998-409e-9117-79ffeaa7d8d7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:29.061969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1efd00-6998-409e-9117-79ffeaa7d8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:29.062245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1efd00-6998-409e-9117-79ffeaa7d8d7 None admin] PATH_INFO: `/v3/users/157afd8d4fea4805a11a42e0c2bcff18` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:29.063681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1efd00-6998-409e-9117-79ffeaa7d8d7 None admin] RBAC: Authorizing `identity:get_user(user_id=157afd8d4fea4805a11a42e0c2bcff18)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:29.065618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1efd00-6998-409e-9117-79ffeaa7d8d7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:29.067512 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 561/1097] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:35:28 2022] GET /identity/v3/users/157afd8d4fea4805a11a42e0c2bcff18 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:29.073506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1efd00-6998-409e-9117-79ffeaa7d8d7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:29.089074 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:29.092065 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21e989-e78e-4157-9714-0e6f1e30d2d8 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:29.126514 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11004b-5ec3-46bd-834a-108dfd91862c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:29.127044 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11004b-5ec3-46bd-834a-108dfd91862c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:29.127314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11004b-5ec3-46bd-834a-108dfd91862c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:29.127579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c11004b-5ec3-46bd-834a-108dfd91862c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:29.172208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1efd00-6998-409e-9117-79ffeaa7d8d7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:29.191829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11004b-5ec3-46bd-834a-108dfd91862c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:29.193529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c11004b-5ec3-46bd-834a-108dfd91862c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:29.204269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd698f2-c7f5-4da0-817e-301a46afde8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:29.204881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd698f2-c7f5-4da0-817e-301a46afde8b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:29.205143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd698f2-c7f5-4da0-817e-301a46afde8b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:29.205408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd698f2-c7f5-4da0-817e-301a46afde8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:29.249074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:29.269891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd698f2-c7f5-4da0-817e-301a46afde8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:29.271302 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd698f2-c7f5-4da0-817e-301a46afde8b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:29.287966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:29.291556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 537/1098] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:35:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 19:35:29.297419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11004b-5ec3-46bd-834a-108dfd91862c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:29.366969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 562/1099] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:35:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 19:35:29.372888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd698f2-c7f5-4da0-817e-301a46afde8b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:29.392695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c11004b-5ec3-46bd-834a-108dfd91862c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:29.426601 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5a670-a938-4925-b25a-5ddeacf83d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:29.427224 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5a670-a938-4925-b25a-5ddeacf83d89 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:29.427499 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5a670-a938-4925-b25a-5ddeacf83d89 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:29.427768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5a670-a938-4925-b25a-5ddeacf83d89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:29.437506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5a670-a938-4925-b25a-5ddeacf83d89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:29.438972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5a670-a938-4925-b25a-5ddeacf83d89 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:29.448033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-98d5a670-a938-4925-b25a-5ddeacf83d89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:35:29.453637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:29.462223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:35:29.463651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:35:29.463651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:35:29.463651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:29.463651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 538/1100] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:29.463651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5a670-a938-4925-b25a-5ddeacf83d89 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:29.469781 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd698f2-c7f5-4da0-817e-301a46afde8b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:29.508766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5349a-e0c9-4365-ae27-e78644729263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:29.512820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5349a-e0c9-4365-ae27-e78644729263 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:29.512820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5349a-e0c9-4365-ae27-e78644729263 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:29.512820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5349a-e0c9-4365-ae27-e78644729263 None admin] PATH_INFO: `/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:29.512820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5349a-e0c9-4365-ae27-e78644729263 None admin] RBAC: Authorizing `identity:get_project(project_id=0f7144ff96fd42378ab24f2bce4c6d78)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:29.513143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5349a-e0c9-4365-ae27-e78644729263 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:29.519601 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 563/1101] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:35:29 2022] GET /identity/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:29.519751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:29.520565 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5349a-e0c9-4365-ae27-e78644729263 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:29.562450 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5a670-a938-4925-b25a-5ddeacf83d89 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:29.596156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3e934-1ec8-4fcc-8f49-577897a95ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:29.596761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3e934-1ec8-4fcc-8f49-577897a95ee6 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:29.597039 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3e934-1ec8-4fcc-8f49-577897a95ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:29.597312 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3e934-1ec8-4fcc-8f49-577897a95ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:29.597981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3e934-1ec8-4fcc-8f49-577897a95ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:29.598281 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3e934-1ec8-4fcc-8f49-577897a95ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:29.599642 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3e934-1ec8-4fcc-8f49-577897a95ee6 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:29.611077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 539/1102] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:29.616745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3e934-1ec8-4fcc-8f49-577897a95ee6 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:29.621709 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5349a-e0c9-4365-ae27-e78644729263 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:29.655813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff574a6-9484-481c-b6c4-bd2dffa4ae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:29.656350 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff574a6-9484-481c-b6c4-bd2dffa4ae60 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:29.656629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff574a6-9484-481c-b6c4-bd2dffa4ae60 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:29.656899 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff574a6-9484-481c-b6c4-bd2dffa4ae60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:29.657608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff574a6-9484-481c-b6c4-bd2dffa4ae60 None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=0f7144ff96fd42378ab24f2bce4c6d78, user.id=157afd8d4fea4805a11a42e0c2bcff18) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:29.657934 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff574a6-9484-481c-b6c4-bd2dffa4ae60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:29.659381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff574a6-9484-481c-b6c4-bd2dffa4ae60 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:29.664903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 564/1103] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:35:29 2022] GET /identity/v3/role_assignments?user.id=157afd8d4fea4805a11a42e0c2bcff18&scope.project.id=0f7144ff96fd42378ab24f2bce4c6d78&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:29.717733 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3e934-1ec8-4fcc-8f49-577897a95ee6 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:29.733074 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:29.754993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec070a50-20b3-46a3-a69f-e174e1f43a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:29.755642 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec070a50-20b3-46a3-a69f-e174e1f43a1a None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:29.755923 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec070a50-20b3-46a3-a69f-e174e1f43a1a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:29.756194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec070a50-20b3-46a3-a69f-e174e1f43a1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:29.764073 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec070a50-20b3-46a3-a69f-e174e1f43a1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:29.765547 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec070a50-20b3-46a3-a69f-e174e1f43a1a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-ec070a50-20b3-46a3-a69f-e174e1f43a1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:35:29.779674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:29.781289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:29.782242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:35:29.782242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:35:29.782242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:35:29.782242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:35:29.782242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:35:29.782242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:35:29.782242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:35:29.782242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:29.782242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 540/1104] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:35:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:35:29.787194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff574a6-9484-481c-b6c4-bd2dffa4ae60 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:29.801228 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:29.889999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff574a6-9484-481c-b6c4-bd2dffa4ae60 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:29.922819 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39a9cc8-3cff-40b0-a275-d0e3f8b6b00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:29.923351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a9cc8-3cff-40b0-a275-d0e3f8b6b00b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:29.923625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a9cc8-3cff-40b0-a275-d0e3f8b6b00b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:29.923899 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39a9cc8-3cff-40b0-a275-d0e3f8b6b00b None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:29.924567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a9cc8-3cff-40b0-a275-d0e3f8b6b00b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:29.924869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a9cc8-3cff-40b0-a275-d0e3f8b6b00b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:29.926312 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39a9cc8-3cff-40b0-a275-d0e3f8b6b00b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:29.937885 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 565/1105] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:35:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:29.944247 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec070a50-20b3-46a3-a69f-e174e1f43a1a None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:30.041385 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec070a50-20b3-46a3-a69f-e174e1f43a1a None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:30.074292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e816f5-eced-4d55-af78-6fc0ef6198f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:30.074889 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e816f5-eced-4d55-af78-6fc0ef6198f3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:30.075163 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e816f5-eced-4d55-af78-6fc0ef6198f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:30.075433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e816f5-eced-4d55-af78-6fc0ef6198f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:30.076140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e816f5-eced-4d55-af78-6fc0ef6198f3 None admin] RBAC: Adding query filter params (role.id=688c3a60f8d34392878a70a5abaffefe, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=2a4284b59f804dc59674d621d87634bc) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:30.076442 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e816f5-eced-4d55-af78-6fc0ef6198f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:30.077889 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e816f5-eced-4d55-af78-6fc0ef6198f3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:30.083475 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 541/1106] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:35:29 2022] GET /identity/v3/role_assignments?user.id=2a4284b59f804dc59674d621d87634bc&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=688c3a60f8d34392878a70a5abaffefe => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:31.847241 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290279d-ab3b-4603-a681-b99bc1469eab None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:31.847774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290279d-ab3b-4603-a681-b99bc1469eab None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:31.847774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290279d-ab3b-4603-a681-b99bc1469eab None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:31.849117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 566/1107] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:31.855024 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0e886-d3e6-47b0-9f44-ccdf00dfab50 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:31.855431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0e886-d3e6-47b0-9f44-ccdf00dfab50 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:31.855567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0e886-d3e6-47b0-9f44-ccdf00dfab50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:31.873543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b8d0e886-d3e6-47b0-9f44-ccdf00dfab50 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:31.874766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b8d0e886-d3e6-47b0-9f44-ccdf00dfab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:31.902663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:31.938968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 542/1108] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:32.205033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290279d-ab3b-4603-a681-b99bc1469eab None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:32.273456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:32.303437 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290279d-ab3b-4603-a681-b99bc1469eab None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:32.336032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077bab2-d46d-4d9e-afcb-742ae9fd644a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:32.336642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077bab2-d46d-4d9e-afcb-742ae9fd644a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:32.336920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077bab2-d46d-4d9e-afcb-742ae9fd644a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:32.337186 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077bab2-d46d-4d9e-afcb-742ae9fd644a None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:32.338887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077bab2-d46d-4d9e-afcb-742ae9fd644a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:32.340293 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077bab2-d46d-4d9e-afcb-742ae9fd644a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:32.341844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 567/1109] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:32.347428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0e886-d3e6-47b0-9f44-ccdf00dfab50 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:32.442221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0e886-d3e6-47b0-9f44-ccdf00dfab50 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:32.474817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a84049-522b-40e8-b6e7-0e7e8f5c4fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:32.475342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a84049-522b-40e8-b6e7-0e7e8f5c4fe3 None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:32.475619 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a84049-522b-40e8-b6e7-0e7e8f5c4fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:32.475889 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a84049-522b-40e8-b6e7-0e7e8f5c4fe3 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:32.476520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a84049-522b-40e8-b6e7-0e7e8f5c4fe3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:32.477953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a84049-522b-40e8-b6e7-0e7e8f5c4fe3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:32.539081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 543/1110] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:35:32 2022] POST /identity/v3/users => generated 286 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:35:34.699134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dde69-73e5-40ce-91d7-28d5c2f35755 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:34.699691 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dde69-73e5-40ce-91d7-28d5c2f35755 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:34.699691 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0dde69-73e5-40ce-91d7-28d5c2f35755 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:34.700969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 568/1111] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:34.706273 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437910b7-adbc-43a2-b922-6d88662235e8 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:34.706671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437910b7-adbc-43a2-b922-6d88662235e8 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:34.706818 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437910b7-adbc-43a2-b922-6d88662235e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:34.724961 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-437910b7-adbc-43a2-b922-6d88662235e8 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:34.726178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-437910b7-adbc-43a2-b922-6d88662235e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:34.742334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:34.789974 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 544/1112] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:35.059215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dde69-73e5-40ce-91d7-28d5c2f35755 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:35.158944 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0dde69-73e5-40ce-91d7-28d5c2f35755 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:35.190789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a19811-1791-45ee-a338-5914c5feaeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:35.191400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a19811-1791-45ee-a338-5914c5feaeb5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:35.191679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a19811-1791-45ee-a338-5914c5feaeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:35.192196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a19811-1791-45ee-a338-5914c5feaeb5 None admin] PATH_INFO: `/v3/roles/36b107b4892f47f8af6b65e1d1d94bb0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:35.193568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a19811-1791-45ee-a338-5914c5feaeb5 None admin] RBAC: Authorizing `identity:get_role(role_id=36b107b4892f47f8af6b65e1d1d94bb0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:35.194974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a19811-1791-45ee-a338-5914c5feaeb5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:35.196251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 569/1113] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:35:35 2022] GET /identity/v3/roles/36b107b4892f47f8af6b65e1d1d94bb0 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:35.202408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437910b7-adbc-43a2-b922-6d88662235e8 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:35.295719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437910b7-adbc-43a2-b922-6d88662235e8 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:35.327904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a5eb1-9def-481d-a1c3-e880979cbf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:35.328453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a5eb1-9def-481d-a1c3-e880979cbf23 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:35.328725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a5eb1-9def-481d-a1c3-e880979cbf23 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:35.328989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a5eb1-9def-481d-a1c3-e880979cbf23 None admin] PATH_INFO: `/v3/users/9bb31ac0e58a4c42a405e1f0c98d98ed` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:35.341814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a5eb1-9def-481d-a1c3-e880979cbf23 None admin] RBAC: Authorizing `identity:get_user(user_id=9bb31ac0e58a4c42a405e1f0c98d98ed)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:35.343232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a5eb1-9def-481d-a1c3-e880979cbf23 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:35.344771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 545/1114] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:35:35 2022] GET /identity/v3/users/9bb31ac0e58a4c42a405e1f0c98d98ed => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:35.350388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a19811-1791-45ee-a338-5914c5feaeb5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:35.359048 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:35.446841 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a19811-1791-45ee-a338-5914c5feaeb5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:35.478929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c67be4-3103-4856-b778-1e7161b93326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:35.479514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c67be4-3103-4856-b778-1e7161b93326 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:35.479798 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c67be4-3103-4856-b778-1e7161b93326 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:35.480070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c67be4-3103-4856-b778-1e7161b93326 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:35.543862 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c67be4-3103-4856-b778-1e7161b93326 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:35.545281 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c67be4-3103-4856-b778-1e7161b93326 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:35.552561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:35.638998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 570/1115] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:35:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:35:35.644725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a5eb1-9def-481d-a1c3-e880979cbf23 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:35.657054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:35.740080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a5eb1-9def-481d-a1c3-e880979cbf23 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:35.772842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a5fdbe-d6b7-4a4a-a960-f212df7530e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:35.773445 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5fdbe-d6b7-4a4a-a960-f212df7530e5 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:35.773723 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5fdbe-d6b7-4a4a-a960-f212df7530e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:35.774014 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a5fdbe-d6b7-4a4a-a960-f212df7530e5 None admin] PATH_INFO: `/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:35.775090 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a5fdbe-d6b7-4a4a-a960-f212df7530e5 None admin] RBAC: Authorizing `identity:get_project(project_id=0f7144ff96fd42378ab24f2bce4c6d78)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:35.776523 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a5fdbe-d6b7-4a4a-a960-f212df7530e5 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:35.778129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 546/1116] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:35:35 2022] GET /identity/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:35.784047 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c67be4-3103-4856-b778-1e7161b93326 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:35.878248 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c67be4-3103-4856-b778-1e7161b93326 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:35.885396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:35.910188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ee65c-f40f-49ef-b790-1301f0330b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:35.910729 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ee65c-f40f-49ef-b790-1301f0330b48 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:35.911004 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ee65c-f40f-49ef-b790-1301f0330b48 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:35.911279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ee65c-f40f-49ef-b790-1301f0330b48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:35.911980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ee65c-f40f-49ef-b790-1301f0330b48 None admin] RBAC: Adding query filter params (role.id=36b107b4892f47f8af6b65e1d1d94bb0, scope.project.id=0f7144ff96fd42378ab24f2bce4c6d78, user.id=9bb31ac0e58a4c42a405e1f0c98d98ed) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:35.912283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ee65c-f40f-49ef-b790-1301f0330b48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:35.913696 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ee65c-f40f-49ef-b790-1301f0330b48 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:35.919641 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 571/1117] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:35:35 2022] GET /identity/v3/role_assignments?user.id=9bb31ac0e58a4c42a405e1f0c98d98ed&scope.project.id=0f7144ff96fd42378ab24f2bce4c6d78&role.id=36b107b4892f47f8af6b65e1d1d94bb0 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:37.988203 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18574668-e465-40fa-b6af-b3df85329d48 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:37.988720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18574668-e465-40fa-b6af-b3df85329d48 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:37.988810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18574668-e465-40fa-b6af-b3df85329d48 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:37.990140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 547/1118] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:37.995831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f13201-3f82-4cd8-81af-99cc911dd5ab None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:37.996210 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f13201-3f82-4cd8-81af-99cc911dd5ab None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:37.996367 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f13201-3f82-4cd8-81af-99cc911dd5ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:38.014639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-28f13201-3f82-4cd8-81af-99cc911dd5ab None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:38.015811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-28f13201-3f82-4cd8-81af-99cc911dd5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:38.080416 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 572/1119] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:38.345599 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18574668-e465-40fa-b6af-b3df85329d48 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:38.421157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:38.442196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18574668-e465-40fa-b6af-b3df85329d48 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:38.474598 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e0ad1-4b85-4f7c-a226-e0cd9928ce31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:38.475200 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e0ad1-4b85-4f7c-a226-e0cd9928ce31 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:38.475471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e0ad1-4b85-4f7c-a226-e0cd9928ce31 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:38.475736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e0ad1-4b85-4f7c-a226-e0cd9928ce31 None admin] PATH_INFO: `/v3/roles/36b107b4892f47f8af6b65e1d1d94bb0` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:38.477084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e0ad1-4b85-4f7c-a226-e0cd9928ce31 None admin] RBAC: Authorizing `identity:get_role(role_id=36b107b4892f47f8af6b65e1d1d94bb0)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:38.479010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e0ad1-4b85-4f7c-a226-e0cd9928ce31 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:38.480281 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 548/1120] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:35:38 2022] GET /identity/v3/roles/36b107b4892f47f8af6b65e1d1d94bb0 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:38.485993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f13201-3f82-4cd8-81af-99cc911dd5ab None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:38.575998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:38.578792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f13201-3f82-4cd8-81af-99cc911dd5ab None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:38.611452 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc50a6c-9e6e-432a-ab19-e03597aa7818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:38.612006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc50a6c-9e6e-432a-ab19-e03597aa7818 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:38.612288 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc50a6c-9e6e-432a-ab19-e03597aa7818 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:38.612557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc50a6c-9e6e-432a-ab19-e03597aa7818 None admin] PATH_INFO: `/v3/users/9bb31ac0e58a4c42a405e1f0c98d98ed` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:38.613646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc50a6c-9e6e-432a-ab19-e03597aa7818 None admin] RBAC: Authorizing `identity:get_user(user_id=9bb31ac0e58a4c42a405e1f0c98d98ed)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:38.615084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc50a6c-9e6e-432a-ab19-e03597aa7818 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:38.616611 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 573/1121] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:35:38 2022] GET /identity/v3/users/9bb31ac0e58a4c42a405e1f0c98d98ed => generated 286 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:38.622187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e0ad1-4b85-4f7c-a226-e0cd9928ce31 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:38.716438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e0ad1-4b85-4f7c-a226-e0cd9928ce31 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:38.748504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f719f281-826a-4761-a50e-34daf3e8c93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:38.749107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719f281-826a-4761-a50e-34daf3e8c93d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:38.749383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719f281-826a-4761-a50e-34daf3e8c93d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:38.749651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719f281-826a-4761-a50e-34daf3e8c93d None admin] PATH_INFO: `/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:38.750766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719f281-826a-4761-a50e-34daf3e8c93d None admin] RBAC: Authorizing `identity:get_project(project_id=0f7144ff96fd42378ab24f2bce4c6d78)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:38.752168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f719f281-826a-4761-a50e-34daf3e8c93d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:38.753737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 549/1122] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:35:38 2022] GET /identity/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:38.759319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc50a6c-9e6e-432a-ab19-e03597aa7818 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:38.853925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc50a6c-9e6e-432a-ab19-e03597aa7818 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:38.886335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acf9036-55ae-4bee-814c-7b23838f0b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:38.886981 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf9036-55ae-4bee-814c-7b23838f0b02 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:38.887257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf9036-55ae-4bee-814c-7b23838f0b02 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:38.887529 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acf9036-55ae-4bee-814c-7b23838f0b02 None admin] PATH_INFO: `/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78/users/9bb31ac0e58a4c42a405e1f0c98d98ed/roles/36b107b4892f47f8af6b65e1d1d94bb0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:38.889629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf9036-55ae-4bee-814c-7b23838f0b02 None admin] RBAC: Authorizing `identity:create_grant(project_id=0f7144ff96fd42378ab24f2bce4c6d78, user_id=9bb31ac0e58a4c42a405e1f0c98d98ed, role_id=36b107b4892f47f8af6b65e1d1d94bb0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:38.891077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acf9036-55ae-4bee-814c-7b23838f0b02 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:38.960865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 574/1123] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:35:38 2022] PUT /identity/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78/users/9bb31ac0e58a4c42a405e1f0c98d98ed/roles/36b107b4892f47f8af6b65e1d1d94bb0 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:35:41.050646 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67a0ce-4b90-447a-bea9-1b81f18ce4b5 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:41.051164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67a0ce-4b90-447a-bea9-1b81f18ce4b5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:41.051164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67a0ce-4b90-447a-bea9-1b81f18ce4b5 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:41.052496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 550/1124] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:41.057820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742dff5-11c2-457b-ae17-16fbc086911c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:41.058200 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742dff5-11c2-457b-ae17-16fbc086911c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:41.058372 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742dff5-11c2-457b-ae17-16fbc086911c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:41.076787 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0742dff5-11c2-457b-ae17-16fbc086911c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:41.078001 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0742dff5-11c2-457b-ae17-16fbc086911c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:41.100164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:41.141679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 575/1125] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:41.408110 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67a0ce-4b90-447a-bea9-1b81f18ce4b5 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:41.423261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:41.505087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67a0ce-4b90-447a-bea9-1b81f18ce4b5 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:41.537290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91bbe2-6f78-46ff-af9f-25c20714cb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:41.537917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91bbe2-6f78-46ff-af9f-25c20714cb99 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:41.538192 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91bbe2-6f78-46ff-af9f-25c20714cb99 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:41.538456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c91bbe2-6f78-46ff-af9f-25c20714cb99 None admin] PATH_INFO: `/v3/roles/36b107b4892f47f8af6b65e1d1d94bb0` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:41.539806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91bbe2-6f78-46ff-af9f-25c20714cb99 None admin] RBAC: Authorizing `identity:get_role(role_id=36b107b4892f47f8af6b65e1d1d94bb0)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:41.541187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c91bbe2-6f78-46ff-af9f-25c20714cb99 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:41.542563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 551/1126] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:35:41 2022] GET /identity/v3/roles/36b107b4892f47f8af6b65e1d1d94bb0 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:41.548460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742dff5-11c2-457b-ae17-16fbc086911c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:41.643483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742dff5-11c2-457b-ae17-16fbc086911c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:41.675577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cf953-2fe4-449a-9a5e-4ab7dc37f169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:41.676137 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cf953-2fe4-449a-9a5e-4ab7dc37f169 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:41.676413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cf953-2fe4-449a-9a5e-4ab7dc37f169 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:41.676682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538cf953-2fe4-449a-9a5e-4ab7dc37f169 None admin] PATH_INFO: `/v3/users/9bb31ac0e58a4c42a405e1f0c98d98ed` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:41.677777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538cf953-2fe4-449a-9a5e-4ab7dc37f169 None admin] RBAC: Authorizing `identity:get_user(user_id=9bb31ac0e58a4c42a405e1f0c98d98ed)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:41.679232 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538cf953-2fe4-449a-9a5e-4ab7dc37f169 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:41.680761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 576/1127] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:35:41 2022] GET /identity/v3/users/9bb31ac0e58a4c42a405e1f0c98d98ed => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:41.686849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91bbe2-6f78-46ff-af9f-25c20714cb99 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:41.764911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:41.782023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c91bbe2-6f78-46ff-af9f-25c20714cb99 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:41.814476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a167d-6052-4954-8d9f-14dd38c97ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:41.815065 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a167d-6052-4954-8d9f-14dd38c97ac5 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:41.815337 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a167d-6052-4954-8d9f-14dd38c97ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:41.815600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a167d-6052-4954-8d9f-14dd38c97ac5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:41.880595 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a167d-6052-4954-8d9f-14dd38c97ac5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:41.882059 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56a167d-6052-4954-8d9f-14dd38c97ac5 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:41.958306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:41.975502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 552/1128] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:35:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:35:41.981987 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cf953-2fe4-449a-9a5e-4ab7dc37f169 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:42.008123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:42.075459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538cf953-2fe4-449a-9a5e-4ab7dc37f169 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:42.107219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664db64-b387-461b-8205-f19e53cc068d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:42.107825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664db64-b387-461b-8205-f19e53cc068d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:42.108097 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664db64-b387-461b-8205-f19e53cc068d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:42.108364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664db64-b387-461b-8205-f19e53cc068d None admin] PATH_INFO: `/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:42.109445 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664db64-b387-461b-8205-f19e53cc068d None admin] RBAC: Authorizing `identity:get_project(project_id=0f7144ff96fd42378ab24f2bce4c6d78)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:42.110889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664db64-b387-461b-8205-f19e53cc068d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:42.112455 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 577/1129] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:35:41 2022] GET /identity/v3/projects/0f7144ff96fd42378ab24f2bce4c6d78 => generated 311 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:42.118460 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a167d-6052-4954-8d9f-14dd38c97ac5 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:42.211988 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56a167d-6052-4954-8d9f-14dd38c97ac5 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:42.244666 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc88c3-8af1-436e-8381-1b9c2c663b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:42.245246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc88c3-8af1-436e-8381-1b9c2c663b5c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:42.245518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc88c3-8af1-436e-8381-1b9c2c663b5c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:42.245781 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc88c3-8af1-436e-8381-1b9c2c663b5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:42.246495 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc88c3-8af1-436e-8381-1b9c2c663b5c None admin] RBAC: Adding query filter params (role.id=36b107b4892f47f8af6b65e1d1d94bb0, scope.project.id=0f7144ff96fd42378ab24f2bce4c6d78, user.id=9bb31ac0e58a4c42a405e1f0c98d98ed) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:42.246793 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc88c3-8af1-436e-8381-1b9c2c663b5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:42.248178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc88c3-8af1-436e-8381-1b9c2c663b5c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:42.253633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 553/1130] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:35:42 2022] GET /identity/v3/role_assignments?user.id=9bb31ac0e58a4c42a405e1f0c98d98ed&scope.project.id=0f7144ff96fd42378ab24f2bce4c6d78&role.id=36b107b4892f47f8af6b65e1d1d94bb0 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:44.359905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f0a42-2f50-4850-8263-55f46f50b2d6 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:44.360423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f0a42-2f50-4850-8263-55f46f50b2d6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:44.360423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f0a42-2f50-4850-8263-55f46f50b2d6 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:44.361749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 578/1131] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:44.366891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072c607-ee04-47d3-9148-849453167265 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:44.367260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072c607-ee04-47d3-9148-849453167265 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:44.367429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c072c607-ee04-47d3-9148-849453167265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:44.385481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-c072c607-ee04-47d3-9148-849453167265 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:44.386689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c072c607-ee04-47d3-9148-849453167265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:44.426341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:44.450663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 554/1132] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:44.714759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f0a42-2f50-4850-8263-55f46f50b2d6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:44.802705 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:44.810312 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f0a42-2f50-4850-8263-55f46f50b2d6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:44.843200 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bed2c40-86e2-497e-97ed-ba2d6ad66221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:44.843805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed2c40-86e2-497e-97ed-ba2d6ad66221 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:44.844076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed2c40-86e2-497e-97ed-ba2d6ad66221 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:44.844339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bed2c40-86e2-497e-97ed-ba2d6ad66221 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:44.845408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed2c40-86e2-497e-97ed-ba2d6ad66221 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:44.846842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bed2c40-86e2-497e-97ed-ba2d6ad66221 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:44.848345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 579/1133] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:44.854697 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072c607-ee04-47d3-9148-849453167265 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:44.949570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c072c607-ee04-47d3-9148-849453167265 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:44.982023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81ea212-86f0-4d1c-bebd-d931fbddfc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:44.982584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ea212-86f0-4d1c-bebd-d931fbddfc58 None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:44.982856 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ea212-86f0-4d1c-bebd-d931fbddfc58 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:44.983119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81ea212-86f0-4d1c-bebd-d931fbddfc58 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:44.983750 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ea212-86f0-4d1c-bebd-d931fbddfc58 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:44.985158 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81ea212-86f0-4d1c-bebd-d931fbddfc58 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:45.069240 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 555/1134] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 19:35:44 2022] POST /identity/v3/projects => generated 311 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:35:47.191802 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c570b-bfd9-4b42-b342-ebf860874d41 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:47.192567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c570b-bfd9-4b42-b342-ebf860874d41 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:47.192567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c570b-bfd9-4b42-b342-ebf860874d41 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:47.193869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 580/1135] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:47.199983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13484e47-aaca-4a19-b021-432d7f126585 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:47.200404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13484e47-aaca-4a19-b021-432d7f126585 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:47.200527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13484e47-aaca-4a19-b021-432d7f126585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:47.218799 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-13484e47-aaca-4a19-b021-432d7f126585 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:47.219974 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-13484e47-aaca-4a19-b021-432d7f126585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:47.241405 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:47.283450 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 556/1136] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:47.548533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c570b-bfd9-4b42-b342-ebf860874d41 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:47.647567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c570b-bfd9-4b42-b342-ebf860874d41 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:47.680386 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f5fc1-9061-4865-b1d4-1071d4691b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:47.681528 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f5fc1-9061-4865-b1d4-1071d4691b56 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:47.681819 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f5fc1-9061-4865-b1d4-1071d4691b56 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:47.682091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f5fc1-9061-4865-b1d4-1071d4691b56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:47.683179 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f5fc1-9061-4865-b1d4-1071d4691b56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:47.684590 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f5fc1-9061-4865-b1d4-1071d4691b56 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:47.686149 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 581/1137] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:35:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:47.692112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13484e47-aaca-4a19-b021-432d7f126585 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:47.787066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13484e47-aaca-4a19-b021-432d7f126585 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:47.819879 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e550b0-bdb5-48c7-bb97-ede6b8cbdb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:47.820412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e550b0-bdb5-48c7-bb97-ede6b8cbdb4b None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:47.820690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e550b0-bdb5-48c7-bb97-ede6b8cbdb4b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:47.820961 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e550b0-bdb5-48c7-bb97-ede6b8cbdb4b None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:47.822186 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e550b0-bdb5-48c7-bb97-ede6b8cbdb4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:47.823616 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e550b0-bdb5-48c7-bb97-ede6b8cbdb4b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:47.910025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 557/1138] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:35:47 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:35:50.075146 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5b933-25d0-485c-b591-45433ce72ec0 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:50.075664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5b933-25d0-485c-b591-45433ce72ec0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:50.075664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a5b933-25d0-485c-b591-45433ce72ec0 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:50.076981 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 582/1139] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:50.082467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca349d-2ce9-4709-9b27-d73aa856d835 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:50.082884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca349d-2ce9-4709-9b27-d73aa856d835 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:50.083009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca349d-2ce9-4709-9b27-d73aa856d835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:50.101313 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-68ca349d-2ce9-4709-9b27-d73aa856d835 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:50.102527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-68ca349d-2ce9-4709-9b27-d73aa856d835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:50.111463 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:50.166708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 558/1140] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:50.436424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5b933-25d0-485c-b591-45433ce72ec0 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:50.465913 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:50.533642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a5b933-25d0-485c-b591-45433ce72ec0 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:50.566216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f79e18-80a9-4761-9c6e-b44c707401c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:50.567097 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f79e18-80a9-4761-9c6e-b44c707401c7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:50.567369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f79e18-80a9-4761-9c6e-b44c707401c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:50.567632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f79e18-80a9-4761-9c6e-b44c707401c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:50.577409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f79e18-80a9-4761-9c6e-b44c707401c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-90f79e18-80a9-4761-9c6e-b44c707401c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:35:50.582107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:50.583124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:50.583953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:50.583953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:50.583953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:50.583953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:50.583953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:35:50.583953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:35:50.583953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:35:50.583953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:35:50.583953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:35:50.583953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:35:50.583953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:35:50.583953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:50.583953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 583/1141] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:35:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:50.589779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca349d-2ce9-4709-9b27-d73aa856d835 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:50.683944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca349d-2ce9-4709-9b27-d73aa856d835 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:50.716244 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530711a-10a5-4433-8b1c-ed76412869cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:50.716766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530711a-10a5-4433-8b1c-ed76412869cd None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:50.717038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530711a-10a5-4433-8b1c-ed76412869cd None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:50.717301 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c530711a-10a5-4433-8b1c-ed76412869cd None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:50.717999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c530711a-10a5-4433-8b1c-ed76412869cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:50.718293 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c530711a-10a5-4433-8b1c-ed76412869cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:50.719638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c530711a-10a5-4433-8b1c-ed76412869cd None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:50.727381 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 559/1142] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:35:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:50.733839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f79e18-80a9-4761-9c6e-b44c707401c7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:50.820234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:50.829890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f79e18-80a9-4761-9c6e-b44c707401c7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:50.862923 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b047d79-d474-4dd5-bfc2-7afeb98d156b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:50.863542 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b047d79-d474-4dd5-bfc2-7afeb98d156b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:50.863820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b047d79-d474-4dd5-bfc2-7afeb98d156b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:50.864090 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b047d79-d474-4dd5-bfc2-7afeb98d156b None admin] PATH_INFO: `/v3/users/41fa3d8ab5f841989fba32f6a84122c0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:50.877460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b047d79-d474-4dd5-bfc2-7afeb98d156b None admin] RBAC: Authorizing `identity:get_user(user_id=41fa3d8ab5f841989fba32f6a84122c0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:50.878925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b047d79-d474-4dd5-bfc2-7afeb98d156b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:50.880517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 584/1143] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:35:50 2022] GET /identity/v3/users/41fa3d8ab5f841989fba32f6a84122c0 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:50.886323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530711a-10a5-4433-8b1c-ed76412869cd None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:50.895288 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:50.982834 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c530711a-10a5-4433-8b1c-ed76412869cd None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:51.014889 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f3f57-44bb-440a-b2c8-df648efb3f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:51.015479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f3f57-44bb-440a-b2c8-df648efb3f5c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:51.015752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f3f57-44bb-440a-b2c8-df648efb3f5c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:51.016017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373f3f57-44bb-440a-b2c8-df648efb3f5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:51.080279 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f3f57-44bb-440a-b2c8-df648efb3f5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:51.081738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373f3f57-44bb-440a-b2c8-df648efb3f5c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:51.088923 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:51.175730 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 560/1144] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:35:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:35:51.182196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b047d79-d474-4dd5-bfc2-7afeb98d156b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:51.277733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b047d79-d474-4dd5-bfc2-7afeb98d156b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:51.309687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af914a6-a69c-4079-b4c6-bfb13fa6d33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:51.310316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af914a6-a69c-4079-b4c6-bfb13fa6d33e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:51.310591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af914a6-a69c-4079-b4c6-bfb13fa6d33e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:51.310857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af914a6-a69c-4079-b4c6-bfb13fa6d33e None admin] PATH_INFO: `/v3/projects/e159136abe624673a15b6c6bc6fbc430` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:51.311935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af914a6-a69c-4079-b4c6-bfb13fa6d33e None admin] RBAC: Authorizing `identity:get_project(project_id=e159136abe624673a15b6c6bc6fbc430)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:51.313339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af914a6-a69c-4079-b4c6-bfb13fa6d33e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:51.314933 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 585/1145] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:35:51 2022] GET /identity/v3/projects/e159136abe624673a15b6c6bc6fbc430 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:51.321240 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f3f57-44bb-440a-b2c8-df648efb3f5c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:51.415916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373f3f57-44bb-440a-b2c8-df648efb3f5c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:51.422851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:51.449302 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f2ce0-a700-45ee-a568-8adbc74a61e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:51.449907 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f2ce0-a700-45ee-a568-8adbc74a61e1 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:51.460291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f2ce0-a700-45ee-a568-8adbc74a61e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:51.460291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f2ce0-a700-45ee-a568-8adbc74a61e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:51.460291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f2ce0-a700-45ee-a568-8adbc74a61e1 None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=e159136abe624673a15b6c6bc6fbc430, user.id=41fa3d8ab5f841989fba32f6a84122c0) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:51.460291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f2ce0-a700-45ee-a568-8adbc74a61e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:51.460291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f2ce0-a700-45ee-a568-8adbc74a61e1 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:51.460718 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 561/1146] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:35:51 2022] GET /identity/v3/role_assignments?user.id=41fa3d8ab5f841989fba32f6a84122c0&scope.project.id=e159136abe624673a15b6c6bc6fbc430&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:35:53.563273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07852948-4790-4c3c-bb41-71364c357519 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:53.563856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07852948-4790-4c3c-bb41-71364c357519 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:53.563856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07852948-4790-4c3c-bb41-71364c357519 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:53.565124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 586/1147] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:53.570545 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90c216-a28b-4fe5-a19f-55531a2a62dd None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:53.570998 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90c216-a28b-4fe5-a19f-55531a2a62dd None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:53.571146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90c216-a28b-4fe5-a19f-55531a2a62dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:53.589360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-8c90c216-a28b-4fe5-a19f-55531a2a62dd None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:53.590566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8c90c216-a28b-4fe5-a19f-55531a2a62dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:53.654257 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 562/1148] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:53.922381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07852948-4790-4c3c-bb41-71364c357519 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:53.940003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:54.020311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07852948-4790-4c3c-bb41-71364c357519 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:54.053493 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f2c8c-72ab-4257-a24d-9913fd7a82e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:54.054140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f2c8c-72ab-4257-a24d-9913fd7a82e2 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:54.054418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f2c8c-72ab-4257-a24d-9913fd7a82e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:54.054686 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f2c8c-72ab-4257-a24d-9913fd7a82e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:54.064921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f2c8c-72ab-4257-a24d-9913fd7a82e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-9b4f2c8c-72ab-4257-a24d-9913fd7a82e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:35:54.069890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:54.071041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:54.072214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:54.072214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:54.072214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:54.072214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:54.072214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:35:54.072214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:35:54.072214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:35:54.072214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:35:54.072214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:35:54.072214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:35:54.072214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:35:54.072214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:35:54.072214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 587/1149] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:35:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:54.076902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90c216-a28b-4fe5-a19f-55531a2a62dd None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:54.169544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:54.172568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90c216-a28b-4fe5-a19f-55531a2a62dd None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:54.205865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacee1c3-0976-499b-93fc-fed48bd6c794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:54.206452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacee1c3-0976-499b-93fc-fed48bd6c794 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:54.206726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacee1c3-0976-499b-93fc-fed48bd6c794 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:54.206993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacee1c3-0976-499b-93fc-fed48bd6c794 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:54.207643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacee1c3-0976-499b-93fc-fed48bd6c794 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:54.207943 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacee1c3-0976-499b-93fc-fed48bd6c794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:54.209296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacee1c3-0976-499b-93fc-fed48bd6c794 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:54.217473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 563/1150] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:35:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:54.223340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f2c8c-72ab-4257-a24d-9913fd7a82e2 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:54.297267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:54.319145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f2c8c-72ab-4257-a24d-9913fd7a82e2 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:54.351883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf413c-dc80-42b4-b830-23fb9c21dd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:54.352494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf413c-dc80-42b4-b830-23fb9c21dd2e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:54.353413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf413c-dc80-42b4-b830-23fb9c21dd2e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:54.353683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf413c-dc80-42b4-b830-23fb9c21dd2e None admin] PATH_INFO: `/v3/users/41fa3d8ab5f841989fba32f6a84122c0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:54.354805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf413c-dc80-42b4-b830-23fb9c21dd2e None admin] RBAC: Authorizing `identity:get_user(user_id=41fa3d8ab5f841989fba32f6a84122c0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:54.356217 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcf413c-dc80-42b4-b830-23fb9c21dd2e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:54.357753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 588/1151] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:35:54 2022] GET /identity/v3/users/41fa3d8ab5f841989fba32f6a84122c0 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:54.363715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacee1c3-0976-499b-93fc-fed48bd6c794 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:54.457638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacee1c3-0976-499b-93fc-fed48bd6c794 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:54.489622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d6b98-95c0-45fe-8b26-e0a2f4c59436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:54.490263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d6b98-95c0-45fe-8b26-e0a2f4c59436 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:54.490546 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d6b98-95c0-45fe-8b26-e0a2f4c59436 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:54.490813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d6b98-95c0-45fe-8b26-e0a2f4c59436 None admin] PATH_INFO: `/v3/projects/e159136abe624673a15b6c6bc6fbc430` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:54.491909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d6b98-95c0-45fe-8b26-e0a2f4c59436 None admin] RBAC: Authorizing `identity:get_project(project_id=e159136abe624673a15b6c6bc6fbc430)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:54.493315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d6b98-95c0-45fe-8b26-e0a2f4c59436 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:54.494927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 564/1152] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:35:54 2022] GET /identity/v3/projects/e159136abe624673a15b6c6bc6fbc430 => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:54.500619 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf413c-dc80-42b4-b830-23fb9c21dd2e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:54.596409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf413c-dc80-42b4-b830-23fb9c21dd2e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:54.628031 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff87599f-f7fa-4c8f-a304-9221572d3f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:54.628659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87599f-f7fa-4c8f-a304-9221572d3f72 None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:54.628935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87599f-f7fa-4c8f-a304-9221572d3f72 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:54.629203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff87599f-f7fa-4c8f-a304-9221572d3f72 None admin] PATH_INFO: `/v3/projects/e159136abe624673a15b6c6bc6fbc430/users/41fa3d8ab5f841989fba32f6a84122c0/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:54.631076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87599f-f7fa-4c8f-a304-9221572d3f72 None admin] RBAC: Authorizing `identity:create_grant(project_id=e159136abe624673a15b6c6bc6fbc430, user_id=41fa3d8ab5f841989fba32f6a84122c0, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:54.632476 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff87599f-f7fa-4c8f-a304-9221572d3f72 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:54.708534 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 589/1153] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:35:54 2022] PUT /identity/v3/projects/e159136abe624673a15b6c6bc6fbc430/users/41fa3d8ab5f841989fba32f6a84122c0/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:35:56.812168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dd207-97ef-4b84-a560-451d7d43675c None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:56.812799 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dd207-97ef-4b84-a560-451d7d43675c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:56.812799 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729dd207-97ef-4b84-a560-451d7d43675c None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:56.814072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 565/1154] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:35:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:35:56.819502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0bc3b-258b-4291-86c3-24a0d9dc560d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:56.819879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0bc3b-258b-4291-86c3-24a0d9dc560d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:56.820068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0bc3b-258b-4291-86c3-24a0d9dc560d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:56.827833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:56.838575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-13f0bc3b-258b-4291-86c3-24a0d9dc560d None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:35:56.839751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-13f0bc3b-258b-4291-86c3-24a0d9dc560d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:35:56.903630 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 590/1155] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:35:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:35:57.179895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729dd207-97ef-4b84-a560-451d7d43675c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:57.211303 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:57.282347 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729dd207-97ef-4b84-a560-451d7d43675c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:57.314753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedaddf8-019a-42a9-b379-d769b0514220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:57.315366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedaddf8-019a-42a9-b379-d769b0514220 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:57.315644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedaddf8-019a-42a9-b379-d769b0514220 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:57.315917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedaddf8-019a-42a9-b379-d769b0514220 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:57.326448 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedaddf8-019a-42a9-b379-d769b0514220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-aedaddf8-019a-42a9-b379-d769b0514220 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:35:57.331175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:35:57.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:35:57.333131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:35:57.333131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:35:57.333131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:35:57.333131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:35:57.333131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:35:57.333131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:35:57.333131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:35:57.333131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:35:57.333131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:35:57.333131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:35:57.333131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:35:57.333131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:35:57.333131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 566/1156] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:35:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:35:57.338761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0bc3b-258b-4291-86c3-24a0d9dc560d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:57.431339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0bc3b-258b-4291-86c3-24a0d9dc560d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:57.463366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e05f3-f463-417c-a929-aded0d244131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:57.463945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e05f3-f463-417c-a929-aded0d244131 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:57.464217 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e05f3-f463-417c-a929-aded0d244131 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:57.464483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e05f3-f463-417c-a929-aded0d244131 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:57.465118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e05f3-f463-417c-a929-aded0d244131 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:57.465411 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e05f3-f463-417c-a929-aded0d244131 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:57.466779 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e05f3-f463-417c-a929-aded0d244131 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:57.468055 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:57.474819 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 591/1157] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:35:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:57.481310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedaddf8-019a-42a9-b379-d769b0514220 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:57.567424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:57.576957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedaddf8-019a-42a9-b379-d769b0514220 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:57.609091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35ba38-f8b2-47f3-a93a-8ea052755133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:57.609697 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35ba38-f8b2-47f3-a93a-8ea052755133 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:57.610128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35ba38-f8b2-47f3-a93a-8ea052755133 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:57.610403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35ba38-f8b2-47f3-a93a-8ea052755133 None admin] PATH_INFO: `/v3/users/41fa3d8ab5f841989fba32f6a84122c0` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:57.611504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35ba38-f8b2-47f3-a93a-8ea052755133 None admin] RBAC: Authorizing `identity:get_user(user_id=41fa3d8ab5f841989fba32f6a84122c0)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:57.612915 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b35ba38-f8b2-47f3-a93a-8ea052755133 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:57.614493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 567/1158] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:35:57 2022] GET /identity/v3/users/41fa3d8ab5f841989fba32f6a84122c0 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:57.620221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e05f3-f463-417c-a929-aded0d244131 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:57.716119 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e05f3-f463-417c-a929-aded0d244131 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:57.747226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cd254-05d4-4bbc-8fbb-a1b6cc9c5672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:57.747817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cd254-05d4-4bbc-8fbb-a1b6cc9c5672 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:57.748093 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cd254-05d4-4bbc-8fbb-a1b6cc9c5672 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:57.748364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cd254-05d4-4bbc-8fbb-a1b6cc9c5672 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:57.805818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:57.811883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7cd254-05d4-4bbc-8fbb-a1b6cc9c5672 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:57.813310 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7cd254-05d4-4bbc-8fbb-a1b6cc9c5672 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:57.908164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 592/1159] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:35:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:35:57.914167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35ba38-f8b2-47f3-a93a-8ea052755133 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:58.010998 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35ba38-f8b2-47f3-a93a-8ea052755133 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:58.042593 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158652da-597b-44d9-9460-e42afe62e6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:58.043542 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158652da-597b-44d9-9460-e42afe62e6cd None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:58.043823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158652da-597b-44d9-9460-e42afe62e6cd None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:58.044097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158652da-597b-44d9-9460-e42afe62e6cd None admin] PATH_INFO: `/v3/projects/e159136abe624673a15b6c6bc6fbc430` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:58.045204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158652da-597b-44d9-9460-e42afe62e6cd None admin] RBAC: Authorizing `identity:get_project(project_id=e159136abe624673a15b6c6bc6fbc430)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:58.046654 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158652da-597b-44d9-9460-e42afe62e6cd None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:58.048253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 568/1160] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:35:57 2022] GET /identity/v3/projects/e159136abe624673a15b6c6bc6fbc430 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:35:58.054224 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cd254-05d4-4bbc-8fbb-a1b6cc9c5672 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:35:58.142596 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:35:58.148063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cd254-05d4-4bbc-8fbb-a1b6cc9c5672 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:35:58.179614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d6415-a9d8-4051-9ec7-116448535eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:35:58.180199 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d6415-a9d8-4051-9ec7-116448535eee None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:35:58.180982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d6415-a9d8-4051-9ec7-116448535eee None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:35:58.181239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d6415-a9d8-4051-9ec7-116448535eee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:35:58.181972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d6415-a9d8-4051-9ec7-116448535eee None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=e159136abe624673a15b6c6bc6fbc430, user.id=41fa3d8ab5f841989fba32f6a84122c0) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:35:58.182279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d6415-a9d8-4051-9ec7-116448535eee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:35:58.183686 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d6415-a9d8-4051-9ec7-116448535eee None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:35:58.189813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 593/1161] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:35:58 2022] GET /identity/v3/role_assignments?user.id=41fa3d8ab5f841989fba32f6a84122c0&scope.project.id=e159136abe624673a15b6c6bc6fbc430&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:00.321554 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbb01f-ae7e-4eba-9e5a-ca4717961b83 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:00.322123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbb01f-ae7e-4eba-9e5a-ca4717961b83 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:00.322472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabbb01f-ae7e-4eba-9e5a-ca4717961b83 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:00.323803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 569/1162] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:36:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:36:00.329126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112a367-89ee-4592-b763-d4511e596969 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:00.329544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112a367-89ee-4592-b763-d4511e596969 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:00.329651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d112a367-89ee-4592-b763-d4511e596969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:00.347850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d112a367-89ee-4592-b763-d4511e596969 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:36:00.349022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d112a367-89ee-4592-b763-d4511e596969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:36:00.412362 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 594/1163] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:36:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:36:00.677691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbb01f-ae7e-4eba-9e5a-ca4717961b83 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:00.704969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:00.776001 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabbb01f-ae7e-4eba-9e5a-ca4717961b83 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:00.808455 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1b87f-8033-437a-9538-7eff7902248e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:00.809026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1b87f-8033-437a-9538-7eff7902248e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:00.809304 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1b87f-8033-437a-9538-7eff7902248e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:00.809570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1b87f-8033-437a-9538-7eff7902248e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:00.872678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1b87f-8033-437a-9538-7eff7902248e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:00.874147 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1b87f-8033-437a-9538-7eff7902248e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:00.898472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:00.968130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 570/1164] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:36:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:36:00.974095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112a367-89ee-4592-b763-d4511e596969 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:01.060011 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:01.069328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d112a367-89ee-4592-b763-d4511e596969 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:01.101667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2bb27-2446-46d1-8f96-f2568ade33f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:01.102909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2bb27-2446-46d1-8f96-f2568ade33f4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:01.103374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2bb27-2446-46d1-8f96-f2568ade33f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:01.103704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2bb27-2446-46d1-8f96-f2568ade33f4 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:01.110816 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2bb27-2446-46d1-8f96-f2568ade33f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:01.112254 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c2bb27-2446-46d1-8f96-f2568ade33f4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-12c2bb27-2446-46d1-8f96-f2568ade33f4 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:36:01.123343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:36:01.124521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:36:01.125392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 595/1165] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 19:36:00 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:36:01.130384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1b87f-8033-437a-9538-7eff7902248e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:01.225005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1b87f-8033-437a-9538-7eff7902248e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:01.249534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:01.257882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaac48a-256b-423c-808f-29388d20f537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:01.258507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaac48a-256b-423c-808f-29388d20f537 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:01.260902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaac48a-256b-423c-808f-29388d20f537 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:01.260902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaac48a-256b-423c-808f-29388d20f537 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:01.260902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaac48a-256b-423c-808f-29388d20f537 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:36:01.260902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaac48a-256b-423c-808f-29388d20f537 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:01.261430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaac48a-256b-423c-808f-29388d20f537 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:01.268319 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 571/1166] 10.222.0.42 () {56 vars in 1115 bytes} [Wed Sep 14 19:36:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:03.395541 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7f350-bd34-44d3-bc0e-7b546417bd6b None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:03.396072 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7f350-bd34-44d3-bc0e-7b546417bd6b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:03.396072 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7f350-bd34-44d3-bc0e-7b546417bd6b None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:03.397393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 596/1167] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:36:03.402561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fca70-8146-4b15-a5d6-d8ef211aa36e None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:03.402998 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fca70-8146-4b15-a5d6-d8ef211aa36e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:03.403093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568fca70-8146-4b15-a5d6-d8ef211aa36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:03.420633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-568fca70-8146-4b15-a5d6-d8ef211aa36e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:36:03.421813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-568fca70-8146-4b15-a5d6-d8ef211aa36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:36:03.485023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 572/1168] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:36:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:36:03.757759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7f350-bd34-44d3-bc0e-7b546417bd6b None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:03.854197 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7f350-bd34-44d3-bc0e-7b546417bd6b None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:03.886998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4933af56-6319-4453-ac39-136597344979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:03.887591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933af56-6319-4453-ac39-136597344979 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:03.887866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933af56-6319-4453-ac39-136597344979 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:03.888131 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4933af56-6319-4453-ac39-136597344979 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:03.888775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933af56-6319-4453-ac39-136597344979 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:03.890142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4933af56-6319-4453-ac39-136597344979 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:03.972334 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 597/1169] 10.222.0.42 () {60 vars in 1138 bytes} [Wed Sep 14 19:36:03 2022] POST /identity/v3/domains => generated 267 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:36:06.149236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576e684-7ead-4969-89ec-1d2a59d6c2ca None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:06.149751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576e684-7ead-4969-89ec-1d2a59d6c2ca None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:06.149751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576e684-7ead-4969-89ec-1d2a59d6c2ca None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:06.151042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 573/1170] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:36:06.156416 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb79313-79d6-4147-a3f9-e303e7fad871 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:06.156849 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb79313-79d6-4147-a3f9-e303e7fad871 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:06.156953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb79313-79d6-4147-a3f9-e303e7fad871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:06.175055 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4cb79313-79d6-4147-a3f9-e303e7fad871 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:36:06.176214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4cb79313-79d6-4147-a3f9-e303e7fad871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:36:06.178884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:06.242181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 598/1171] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:36:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:36:06.504923 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576e684-7ead-4969-89ec-1d2a59d6c2ca None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:06.601390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576e684-7ead-4969-89ec-1d2a59d6c2ca None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:06.621107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:06.634120 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd8481-ab57-499f-bbf0-ce86e0a6f70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:06.634731 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd8481-ab57-499f-bbf0-ce86e0a6f70a None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:06.635002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd8481-ab57-499f-bbf0-ce86e0a6f70a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:06.635266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd8481-ab57-499f-bbf0-ce86e0a6f70a None admin] PATH_INFO: `/v3/domains/045c5f19d16045c98049d12047454ef4` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:06.644986 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd8481-ab57-499f-bbf0-ce86e0a6f70a None admin] RBAC: Authorizing `identity:get_domain(domain_id=045c5f19d16045c98049d12047454ef4)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:06.646737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd8481-ab57-499f-bbf0-ce86e0a6f70a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:06.648585 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 574/1172] 10.222.0.42 () {56 vars in 1183 bytes} [Wed Sep 14 19:36:06 2022] GET /identity/v3/domains/045c5f19d16045c98049d12047454ef4 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:06.653991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb79313-79d6-4147-a3f9-e303e7fad871 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:06.747474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb79313-79d6-4147-a3f9-e303e7fad871 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:06.778681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb98935-df50-4716-9d84-0f137748337e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:06.779191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb98935-df50-4716-9d84-0f137748337e None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:06.779461 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb98935-df50-4716-9d84-0f137748337e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:06.779723 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb98935-df50-4716-9d84-0f137748337e None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:06.780351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb98935-df50-4716-9d84-0f137748337e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:06.781738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb98935-df50-4716-9d84-0f137748337e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:06.792606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:06.881268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 599/1173] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 19:36:06 2022] POST /identity/v3/projects => generated 361 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:36:09.004845 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e98d7-8fdc-4fbe-a5c6-e9fe9663c798 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:09.005378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e98d7-8fdc-4fbe-a5c6-e9fe9663c798 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:09.005378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162e98d7-8fdc-4fbe-a5c6-e9fe9663c798 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:09.006731 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 575/1174] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:36:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:36:09.012404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9d27c-fbdd-4282-a999-8177df9bffcd None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:09.012834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9d27c-fbdd-4282-a999-8177df9bffcd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:09.012939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9d27c-fbdd-4282-a999-8177df9bffcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:09.031113 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3da9d27c-fbdd-4282-a999-8177df9bffcd None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:36:09.032284 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3da9d27c-fbdd-4282-a999-8177df9bffcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:36:09.096008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 600/1175] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:36:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:36:09.363013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e98d7-8fdc-4fbe-a5c6-e9fe9663c798 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:09.462683 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162e98d7-8fdc-4fbe-a5c6-e9fe9663c798 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:09.494376 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ffcb9-f8fa-4a00-9078-c0e8d71c510c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:09.494980 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ffcb9-f8fa-4a00-9078-c0e8d71c510c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:09.495251 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ffcb9-f8fa-4a00-9078-c0e8d71c510c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:09.495514 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ffcb9-f8fa-4a00-9078-c0e8d71c510c None admin] PATH_INFO: `/v3/domains/045c5f19d16045c98049d12047454ef4` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:09.496597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ffcb9-f8fa-4a00-9078-c0e8d71c510c None admin] RBAC: Authorizing `identity:get_domain(domain_id=045c5f19d16045c98049d12047454ef4)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:09.498034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ffcb9-f8fa-4a00-9078-c0e8d71c510c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:09.499556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 576/1176] 10.222.0.42 () {56 vars in 1183 bytes} [Wed Sep 14 19:36:09 2022] GET /identity/v3/domains/045c5f19d16045c98049d12047454ef4 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:09.505404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9d27c-fbdd-4282-a999-8177df9bffcd None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:09.600369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9d27c-fbdd-4282-a999-8177df9bffcd None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:09.622782 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:09.633600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d4ae3-1fd9-4f31-8273-3fffdc1ea67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:09.640060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d4ae3-1fd9-4f31-8273-3fffdc1ea67d None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:09.640060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d4ae3-1fd9-4f31-8273-3fffdc1ea67d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:09.640060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d4ae3-1fd9-4f31-8273-3fffdc1ea67d None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:09.640060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d4ae3-1fd9-4f31-8273-3fffdc1ea67d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:09.640060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d4ae3-1fd9-4f31-8273-3fffdc1ea67d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:09.759738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 601/1177] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:36:09 2022] POST /identity/v3/users => generated 311 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:36:11.918551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99cd5a-cc14-40ac-a11b-967c1f7bc97c None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:11.919021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99cd5a-cc14-40ac-a11b-967c1f7bc97c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:11.919113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb99cd5a-cc14-40ac-a11b-967c1f7bc97c None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:11.920405 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 577/1178] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:36:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:36:11.925634 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1caf2-9ff2-45b0-bd00-a6ef59a81015 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:11.926074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1caf2-9ff2-45b0-bd00-a6ef59a81015 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:11.926214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b1caf2-9ff2-45b0-bd00-a6ef59a81015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:11.944394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-09b1caf2-9ff2-45b0-bd00-a6ef59a81015 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:36:11.946097 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-09b1caf2-9ff2-45b0-bd00-a6ef59a81015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:36:12.009889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 602/1179] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:36:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:36:12.276933 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99cd5a-cc14-40ac-a11b-967c1f7bc97c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:12.328875 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:12.381610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb99cd5a-cc14-40ac-a11b-967c1f7bc97c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:12.414617 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e470e64-98cc-4939-8a93-7cb7e949d969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:12.415239 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e470e64-98cc-4939-8a93-7cb7e949d969 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:12.415513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e470e64-98cc-4939-8a93-7cb7e949d969 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:12.415780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e470e64-98cc-4939-8a93-7cb7e949d969 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:12.426541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e470e64-98cc-4939-8a93-7cb7e949d969 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-9e470e64-98cc-4939-8a93-7cb7e949d969 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:36:12.431786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:36:12.432895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:36:12.433753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:36:12.433753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:36:12.433753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:36:12.433753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:36:12.433753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:36:12.433753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:36:12.433753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:36:12.433753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:36:12.433753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:36:12.433753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:36:12.433753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:36:12.433753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:36:12.433753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 578/1180] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:36:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:36:12.438983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1caf2-9ff2-45b0-bd00-a6ef59a81015 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:12.541404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b1caf2-9ff2-45b0-bd00-a6ef59a81015 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:12.550195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:12.573863 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56bd78-7893-4b04-b4ae-0ca5b7b235b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:12.574464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56bd78-7893-4b04-b4ae-0ca5b7b235b2 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:12.574739 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56bd78-7893-4b04-b4ae-0ca5b7b235b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:12.575007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56bd78-7893-4b04-b4ae-0ca5b7b235b2 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:12.575665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56bd78-7893-4b04-b4ae-0ca5b7b235b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:36:12.576190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56bd78-7893-4b04-b4ae-0ca5b7b235b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:12.577677 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c56bd78-7893-4b04-b4ae-0ca5b7b235b2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:12.585629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 603/1181] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:36:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:12.592306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e470e64-98cc-4939-8a93-7cb7e949d969 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:12.752532 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e470e64-98cc-4939-8a93-7cb7e949d969 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:12.761637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:12.785977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f00c4-0e8f-43ab-84fd-86ff688371fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:12.786589 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f00c4-0e8f-43ab-84fd-86ff688371fa None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:12.786866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f00c4-0e8f-43ab-84fd-86ff688371fa None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:12.787173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f00c4-0e8f-43ab-84fd-86ff688371fa None admin] PATH_INFO: `/v3/users/998256e3338d4f3695fd8dc6f3ff5bb4` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:12.800024 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f00c4-0e8f-43ab-84fd-86ff688371fa None admin] RBAC: Authorizing `identity:get_user(user_id=998256e3338d4f3695fd8dc6f3ff5bb4)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:12.801725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f00c4-0e8f-43ab-84fd-86ff688371fa None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:12.803323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 579/1182] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:36:12 2022] GET /identity/v3/users/998256e3338d4f3695fd8dc6f3ff5bb4 => generated 311 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:12.809144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56bd78-7893-4b04-b4ae-0ca5b7b235b2 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:12.907952 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56bd78-7893-4b04-b4ae-0ca5b7b235b2 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:12.940012 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ddd78-32cd-46aa-8cb3-c8d460bbd0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:12.940604 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ddd78-32cd-46aa-8cb3-c8d460bbd0af None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:12.940989 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ddd78-32cd-46aa-8cb3-c8d460bbd0af None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:12.941261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77ddd78-32cd-46aa-8cb3-c8d460bbd0af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:12.978671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:13.005265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ddd78-32cd-46aa-8cb3-c8d460bbd0af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:13.006715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77ddd78-32cd-46aa-8cb3-c8d460bbd0af None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:13.099905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 604/1183] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:36:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:36:13.105971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f00c4-0e8f-43ab-84fd-86ff688371fa None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:13.201725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f00c4-0e8f-43ab-84fd-86ff688371fa None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:13.233970 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306457b1-6054-473a-b339-8fd3c1ab2d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:13.234582 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306457b1-6054-473a-b339-8fd3c1ab2d13 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:13.234861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306457b1-6054-473a-b339-8fd3c1ab2d13 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:13.235133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306457b1-6054-473a-b339-8fd3c1ab2d13 None admin] PATH_INFO: `/v3/projects/e3b569f86187449db1a3425c1ed0c91c` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:13.236242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306457b1-6054-473a-b339-8fd3c1ab2d13 None admin] RBAC: Authorizing `identity:get_project(project_id=e3b569f86187449db1a3425c1ed0c91c)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:13.237656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306457b1-6054-473a-b339-8fd3c1ab2d13 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:13.239301 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 580/1184] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:36:13 2022] GET /identity/v3/projects/e3b569f86187449db1a3425c1ed0c91c => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:13.245871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ddd78-32cd-46aa-8cb3-c8d460bbd0af None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:13.310485 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:13.339402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77ddd78-32cd-46aa-8cb3-c8d460bbd0af None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:13.374055 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97514c-0df0-4b92-8e41-4f6d402ce6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:13.374593 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97514c-0df0-4b92-8e41-4f6d402ce6a5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:13.374867 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97514c-0df0-4b92-8e41-4f6d402ce6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:13.375135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97514c-0df0-4b92-8e41-4f6d402ce6a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:13.375838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97514c-0df0-4b92-8e41-4f6d402ce6a5 None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=e3b569f86187449db1a3425c1ed0c91c, user.id=998256e3338d4f3695fd8dc6f3ff5bb4) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:36:13.376137 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97514c-0df0-4b92-8e41-4f6d402ce6a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:13.377539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97514c-0df0-4b92-8e41-4f6d402ce6a5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:13.382735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 605/1185] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:36:13 2022] GET /identity/v3/role_assignments?user.id=998256e3338d4f3695fd8dc6f3ff5bb4&scope.project.id=e3b569f86187449db1a3425c1ed0c91c&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:15.479189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc070a-834b-4366-8cdb-f5b02520eaea None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:15.479741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc070a-834b-4366-8cdb-f5b02520eaea None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:15.479741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cc070a-834b-4366-8cdb-f5b02520eaea None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:15.481059 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 581/1186] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:36:15.486306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488960a-d3d4-4c45-ad2f-f856d279f5d4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:15.486678 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488960a-d3d4-4c45-ad2f-f856d279f5d4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:15.486850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3488960a-d3d4-4c45-ad2f-f856d279f5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:15.505022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3488960a-d3d4-4c45-ad2f-f856d279f5d4 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:36:15.506237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3488960a-d3d4-4c45-ad2f-f856d279f5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:36:15.570112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 606/1187] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:36:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:36:15.840782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc070a-834b-4366-8cdb-f5b02520eaea None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:15.876174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:15.940037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cc070a-834b-4366-8cdb-f5b02520eaea None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:15.972706 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd3fa9-db07-45b7-b8a3-3cd92685ae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:15.973315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd3fa9-db07-45b7-b8a3-3cd92685ae80 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:15.973587 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd3fa9-db07-45b7-b8a3-3cd92685ae80 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:15.973878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bd3fa9-db07-45b7-b8a3-3cd92685ae80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:15.984020 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bd3fa9-db07-45b7-b8a3-3cd92685ae80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-94bd3fa9-db07-45b7-b8a3-3cd92685ae80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:36:15.988690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:36:15.989709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:36:15.990610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:36:15.990610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:36:15.990610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:36:15.990610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:36:15.990610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:36:15.990610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:36:15.990610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:36:15.990610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:36:15.990610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:36:15.990610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:36:15.990610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:36:15.990610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:36:15.990610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 582/1188] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:36:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:36:15.995789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488960a-d3d4-4c45-ad2f-f856d279f5d4 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:16.053896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:16.090478 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3488960a-d3d4-4c45-ad2f-f856d279f5d4 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:16.122329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69e6ef-a297-4925-9532-38d4ba452103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:16.122859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69e6ef-a297-4925-9532-38d4ba452103 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:16.123134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69e6ef-a297-4925-9532-38d4ba452103 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:16.123401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e69e6ef-a297-4925-9532-38d4ba452103 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:16.124035 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e69e6ef-a297-4925-9532-38d4ba452103 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:36:16.124330 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e69e6ef-a297-4925-9532-38d4ba452103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:16.125668 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e69e6ef-a297-4925-9532-38d4ba452103 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:16.133763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 607/1189] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:36:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:16.139543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd3fa9-db07-45b7-b8a3-3cd92685ae80 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:16.230173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:16.236617 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bd3fa9-db07-45b7-b8a3-3cd92685ae80 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:16.269651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfc131-19a1-4e53-8e0b-e09f8001704d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:16.270286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfc131-19a1-4e53-8e0b-e09f8001704d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:16.270561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfc131-19a1-4e53-8e0b-e09f8001704d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:16.270830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfc131-19a1-4e53-8e0b-e09f8001704d None admin] PATH_INFO: `/v3/users/998256e3338d4f3695fd8dc6f3ff5bb4` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:16.271938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfc131-19a1-4e53-8e0b-e09f8001704d None admin] RBAC: Authorizing `identity:get_user(user_id=998256e3338d4f3695fd8dc6f3ff5bb4)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:16.273437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbfc131-19a1-4e53-8e0b-e09f8001704d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:16.275044 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 583/1190] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:36:16 2022] GET /identity/v3/users/998256e3338d4f3695fd8dc6f3ff5bb4 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:16.280878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69e6ef-a297-4925-9532-38d4ba452103 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:16.376733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e69e6ef-a297-4925-9532-38d4ba452103 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:16.399880 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:16.407911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc04e2f1-5625-4d18-8ea4-d4c2ad676304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:16.408520 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04e2f1-5625-4d18-8ea4-d4c2ad676304 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:16.408792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04e2f1-5625-4d18-8ea4-d4c2ad676304 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:16.412311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc04e2f1-5625-4d18-8ea4-d4c2ad676304 None admin] PATH_INFO: `/v3/projects/e3b569f86187449db1a3425c1ed0c91c` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:16.412311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04e2f1-5625-4d18-8ea4-d4c2ad676304 None admin] RBAC: Authorizing `identity:get_project(project_id=e3b569f86187449db1a3425c1ed0c91c)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:16.412311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc04e2f1-5625-4d18-8ea4-d4c2ad676304 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:16.413145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 608/1191] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:36:16 2022] GET /identity/v3/projects/e3b569f86187449db1a3425c1ed0c91c => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:16.419608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfc131-19a1-4e53-8e0b-e09f8001704d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:16.514918 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfc131-19a1-4e53-8e0b-e09f8001704d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:16.547366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cc0fe-f1a0-4025-b992-fb935eb97d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:16.547999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cc0fe-f1a0-4025-b992-fb935eb97d47 None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:16.548275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cc0fe-f1a0-4025-b992-fb935eb97d47 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:16.548543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cc0fe-f1a0-4025-b992-fb935eb97d47 None admin] PATH_INFO: `/v3/projects/e3b569f86187449db1a3425c1ed0c91c/users/998256e3338d4f3695fd8dc6f3ff5bb4/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:16.550433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cc0fe-f1a0-4025-b992-fb935eb97d47 None admin] RBAC: Authorizing `identity:create_grant(project_id=e3b569f86187449db1a3425c1ed0c91c, user_id=998256e3338d4f3695fd8dc6f3ff5bb4, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:16.551844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cc0fe-f1a0-4025-b992-fb935eb97d47 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:16.633618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 584/1192] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:36:16 2022] PUT /identity/v3/projects/e3b569f86187449db1a3425c1ed0c91c/users/998256e3338d4f3695fd8dc6f3ff5bb4/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:36:18.733004 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dbdd6-cc66-43a7-aa85-b52bc4b08d02 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:18.733536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dbdd6-cc66-43a7-aa85-b52bc4b08d02 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:18.733536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408dbdd6-cc66-43a7-aa85-b52bc4b08d02 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:18.734934 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 609/1193] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:36:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:36:18.740412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60358c7d-677d-4bf3-8f5f-55c35aa6e34c None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:18.740835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60358c7d-677d-4bf3-8f5f-55c35aa6e34c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:18.740944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60358c7d-677d-4bf3-8f5f-55c35aa6e34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:18.759176 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-60358c7d-677d-4bf3-8f5f-55c35aa6e34c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:36:18.760346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-60358c7d-677d-4bf3-8f5f-55c35aa6e34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:36:18.775949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:18.824119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 585/1194] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:36:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:36:19.092183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408dbdd6-cc66-43a7-aa85-b52bc4b08d02 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:19.189226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408dbdd6-cc66-43a7-aa85-b52bc4b08d02 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:19.221224 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3df6d-cdcc-44c6-87e5-219b471b634b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:19.221851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3df6d-cdcc-44c6-87e5-219b471b634b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:19.222130 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3df6d-cdcc-44c6-87e5-219b471b634b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:19.222399 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f3df6d-cdcc-44c6-87e5-219b471b634b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:19.232511 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f3df6d-cdcc-44c6-87e5-219b471b634b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-e2f3df6d-cdcc-44c6-87e5-219b471b634b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:36:19.237181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:36:19.238294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:36:19.239524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:36:19.239524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:36:19.239524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:36:19.239524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:36:19.239524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:36:19.239524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:36:19.239524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:36:19.239524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:36:19.239524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:36:19.239524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:36:19.239524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:36:19.239524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:36:19.239524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 610/1195] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:36:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:36:19.244600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60358c7d-677d-4bf3-8f5f-55c35aa6e34c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:19.338164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60358c7d-677d-4bf3-8f5f-55c35aa6e34c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:19.370954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39fdd39-39c1-4535-9c8d-f0745fc5d0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:19.371479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39fdd39-39c1-4535-9c8d-f0745fc5d0fb None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:19.371752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39fdd39-39c1-4535-9c8d-f0745fc5d0fb None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:19.372017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39fdd39-39c1-4535-9c8d-f0745fc5d0fb None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:19.372654 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39fdd39-39c1-4535-9c8d-f0745fc5d0fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:36:19.372948 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39fdd39-39c1-4535-9c8d-f0745fc5d0fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:19.374322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39fdd39-39c1-4535-9c8d-f0745fc5d0fb None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:19.382234 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 586/1196] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:36:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:19.388126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3df6d-cdcc-44c6-87e5-219b471b634b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:19.442907 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:19.485096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f3df6d-cdcc-44c6-87e5-219b471b634b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:19.518190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fef7e-2933-45fb-b5bd-edbb7e941d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:19.518798 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fef7e-2933-45fb-b5bd-edbb7e941d24 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:19.519070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fef7e-2933-45fb-b5bd-edbb7e941d24 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:19.519336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2fef7e-2933-45fb-b5bd-edbb7e941d24 None admin] PATH_INFO: `/v3/users/998256e3338d4f3695fd8dc6f3ff5bb4` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:19.520423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fef7e-2933-45fb-b5bd-edbb7e941d24 None admin] RBAC: Authorizing `identity:get_user(user_id=998256e3338d4f3695fd8dc6f3ff5bb4)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:19.521897 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2fef7e-2933-45fb-b5bd-edbb7e941d24 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:19.523437 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 611/1197] 10.222.0.42 () {56 vars in 1177 bytes} [Wed Sep 14 19:36:19 2022] GET /identity/v3/users/998256e3338d4f3695fd8dc6f3ff5bb4 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:19.529328 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39fdd39-39c1-4535-9c8d-f0745fc5d0fb None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:19.545306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:19.625452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39fdd39-39c1-4535-9c8d-f0745fc5d0fb None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:19.656538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62faa6a-218a-44b0-806d-8c7c23fd0f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:19.657120 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62faa6a-218a-44b0-806d-8c7c23fd0f84 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:19.657393 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62faa6a-218a-44b0-806d-8c7c23fd0f84 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:19.657656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62faa6a-218a-44b0-806d-8c7c23fd0f84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:19.721437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62faa6a-218a-44b0-806d-8c7c23fd0f84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:19.722882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62faa6a-218a-44b0-806d-8c7c23fd0f84 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:19.736901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:19.820027 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 587/1198] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:36:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:36:19.826215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fef7e-2933-45fb-b5bd-edbb7e941d24 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:19.922572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2fef7e-2933-45fb-b5bd-edbb7e941d24 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:19.943743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:19.954641 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a6987-502f-411c-8f32-2257fb89d8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:19.955263 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a6987-502f-411c-8f32-2257fb89d8d2 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:19.955543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a6987-502f-411c-8f32-2257fb89d8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:19.955816 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a6987-502f-411c-8f32-2257fb89d8d2 None admin] PATH_INFO: `/v3/projects/e3b569f86187449db1a3425c1ed0c91c` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:19.956911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a6987-502f-411c-8f32-2257fb89d8d2 None admin] RBAC: Authorizing `identity:get_project(project_id=e3b569f86187449db1a3425c1ed0c91c)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:19.958385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a6987-502f-411c-8f32-2257fb89d8d2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:19.959987 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 612/1199] 10.222.0.42 () {56 vars in 1186 bytes} [Wed Sep 14 19:36:19 2022] GET /identity/v3/projects/e3b569f86187449db1a3425c1ed0c91c => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:19.966673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62faa6a-218a-44b0-806d-8c7c23fd0f84 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:36:20.060778 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62faa6a-218a-44b0-806d-8c7c23fd0f84 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:36:20.083027 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:20.102554 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151b06d2-3570-49b0-aff0-245d2443c74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:36:20.103143 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b06d2-3570-49b0-aff0-245d2443c74a None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:36:20.103414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b06d2-3570-49b0-aff0-245d2443c74a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:36:20.103674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b06d2-3570-49b0-aff0-245d2443c74a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:36:20.104375 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b06d2-3570-49b0-aff0-245d2443c74a None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=e3b569f86187449db1a3425c1ed0c91c, user.id=998256e3338d4f3695fd8dc6f3ff5bb4) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:36:20.104672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b06d2-3570-49b0-aff0-245d2443c74a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:36:20.106146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151b06d2-3570-49b0-aff0-245d2443c74a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:36:20.111609 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 588/1200] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:36:19 2022] GET /identity/v3/role_assignments?user.id=998256e3338d4f3695fd8dc6f3ff5bb4&scope.project.id=e3b569f86187449db1a3425c1ed0c91c&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:36:29.976513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:36:30.113726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=11 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:36.172840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33394d87-a98a-4fe8-a33c-081c73047798 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:36.173393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33394d87-a98a-4fe8-a33c-081c73047798 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:36.173393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33394d87-a98a-4fe8-a33c-081c73047798 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:36.174676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 613/1201] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:37:36.180562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8b19f-06ee-4aac-852d-7a4efbfae9e3 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:36.181057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8b19f-06ee-4aac-852d-7a4efbfae9e3 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:36.181156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8b19f-06ee-4aac-852d-7a4efbfae9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:36.199868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-82b8b19f-06ee-4aac-852d-7a4efbfae9e3 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:37:36.201100 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-82b8b19f-06ee-4aac-852d-7a4efbfae9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:37:36.266673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 589/1202] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:37:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 19:37:36.536863 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33394d87-a98a-4fe8-a33c-081c73047798 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:36.637095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33394d87-a98a-4fe8-a33c-081c73047798 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:36.670170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee81fed-7d49-4dce-8159-910d311a81e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:36.676031 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee81fed-7d49-4dce-8159-910d311a81e2 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:36.676031 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee81fed-7d49-4dce-8159-910d311a81e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:36.676031 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee81fed-7d49-4dce-8159-910d311a81e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:36.682154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee81fed-7d49-4dce-8159-910d311a81e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:36.683577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee81fed-7d49-4dce-8159-910d311a81e2 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-5ee81fed-7d49-4dce-8159-910d311a81e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:37:36.697566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:36.698766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:37:36.699909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:37:36.699909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:36.699909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:37:36.699909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 614/1203] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:36.704900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8b19f-06ee-4aac-852d-7a4efbfae9e3 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:36.800270 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8b19f-06ee-4aac-852d-7a4efbfae9e3 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:36.836151 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:36.841767 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed299039-9cfa-4f21-a450-7c3ba3ae3026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:36.842396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed299039-9cfa-4f21-a450-7c3ba3ae3026 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:36.842671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed299039-9cfa-4f21-a450-7c3ba3ae3026 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:36.842934 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed299039-9cfa-4f21-a450-7c3ba3ae3026 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:36.843557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed299039-9cfa-4f21-a450-7c3ba3ae3026 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:36.843862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed299039-9cfa-4f21-a450-7c3ba3ae3026 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:36.845200 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed299039-9cfa-4f21-a450-7c3ba3ae3026 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:36.856572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 590/1204] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:37:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:36.862277 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee81fed-7d49-4dce-8159-910d311a81e2 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:36.915162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:36.961325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee81fed-7d49-4dce-8159-910d311a81e2 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:36.993345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa328c-a557-4bd8-a6e3-40f45a784b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:36.993969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa328c-a557-4bd8-a6e3-40f45a784b83 None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:36.994252 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa328c-a557-4bd8-a6e3-40f45a784b83 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:36.994518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa328c-a557-4bd8-a6e3-40f45a784b83 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:36.995148 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa328c-a557-4bd8-a6e3-40f45a784b83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:36.996550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa328c-a557-4bd8-a6e3-40f45a784b83 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:37.099095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 615/1205] 10.222.0.42 () {60 vars in 1132 bytes} [Wed Sep 14 19:37:36 2022] POST /identity/v3/users => generated 251 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:37:39.291307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd0dc7-5c1e-4fb0-919f-8cac29ff5019 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:39.291855 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd0dc7-5c1e-4fb0-919f-8cac29ff5019 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:39.291855 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd0dc7-5c1e-4fb0-919f-8cac29ff5019 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:39.293132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 591/1206] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:37:39.298538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbf2fb-6f11-40e0-b751-9278f038bc48 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:39.303209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbf2fb-6f11-40e0-b751-9278f038bc48 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:39.303209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bbf2fb-6f11-40e0-b751-9278f038bc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:39.317238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c7bbf2fb-6f11-40e0-b751-9278f038bc48 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:37:39.318437 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c7bbf2fb-6f11-40e0-b751-9278f038bc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:37:39.382764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 616/1207] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:37:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:37:39.650508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd0dc7-5c1e-4fb0-919f-8cac29ff5019 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:39.751453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd0dc7-5c1e-4fb0-919f-8cac29ff5019 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:39.784506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11019b20-9c40-4b49-b1df-2c8b64255fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:39.785120 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11019b20-9c40-4b49-b1df-2c8b64255fc9 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:39.785395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11019b20-9c40-4b49-b1df-2c8b64255fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:39.785662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11019b20-9c40-4b49-b1df-2c8b64255fc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:39.795815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11019b20-9c40-4b49-b1df-2c8b64255fc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-11019b20-9c40-4b49-b1df-2c8b64255fc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:37:39.800482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:39.801624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:39.802563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:39.802563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:39.802563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:39.802563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:39.802563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:37:39.802563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:37:39.802563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:37:39.802563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:37:39.802563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:37:39.802563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:37:39.802563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:37:39.802563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:37:39.802563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 592/1208] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:37:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:39.807592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbf2fb-6f11-40e0-b751-9278f038bc48 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:39.847344 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:39.904940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bbf2fb-6f11-40e0-b751-9278f038bc48 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:39.937779 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c64d8-97ec-4c8b-a249-91ff76b52e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:39.938404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c64d8-97ec-4c8b-a249-91ff76b52e9e None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:39.938679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c64d8-97ec-4c8b-a249-91ff76b52e9e None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:39.938949 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c64d8-97ec-4c8b-a249-91ff76b52e9e None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:39.939597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c64d8-97ec-4c8b-a249-91ff76b52e9e None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:39.939895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c64d8-97ec-4c8b-a249-91ff76b52e9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:39.941513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c64d8-97ec-4c8b-a249-91ff76b52e9e None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:39.949505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 617/1209] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:37:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:39.955361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11019b20-9c40-4b49-b1df-2c8b64255fc9 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:40.004817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:40.052574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11019b20-9c40-4b49-b1df-2c8b64255fc9 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:40.085279 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d35a4-90bf-4911-b588-7f0dd6b31a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:40.085912 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d35a4-90bf-4911-b588-7f0dd6b31a7d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:40.086236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d35a4-90bf-4911-b588-7f0dd6b31a7d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:40.086506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d35a4-90bf-4911-b588-7f0dd6b31a7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:40.097425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d35a4-90bf-4911-b588-7f0dd6b31a7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:40.098886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d35a4-90bf-4911-b588-7f0dd6b31a7d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-3a1d35a4-90bf-4911-b588-7f0dd6b31a7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:37:40.112552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:40.113606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:37:40.114560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:37:40.114560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:40.114560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:37:40.114560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 593/1210] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:40.119953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c64d8-97ec-4c8b-a249-91ff76b52e9e None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:40.215119 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:40.217940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c64d8-97ec-4c8b-a249-91ff76b52e9e None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:40.250156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8c308-3df6-4158-9a91-18b19cbd18ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:40.250752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8c308-3df6-4158-9a91-18b19cbd18ed None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:40.251035 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8c308-3df6-4158-9a91-18b19cbd18ed None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:40.251304 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e8c308-3df6-4158-9a91-18b19cbd18ed None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:40.252001 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8c308-3df6-4158-9a91-18b19cbd18ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:40.252314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8c308-3df6-4158-9a91-18b19cbd18ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:40.253671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e8c308-3df6-4158-9a91-18b19cbd18ed None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:40.265546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 618/1211] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:40.271413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d35a4-90bf-4911-b588-7f0dd6b31a7d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:40.369333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d35a4-90bf-4911-b588-7f0dd6b31a7d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:40.370940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:40.402145 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796202c-a57a-4e65-8ec6-ed9d383b2893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:40.402815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796202c-a57a-4e65-8ec6-ed9d383b2893 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:40.403087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796202c-a57a-4e65-8ec6-ed9d383b2893 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:40.403351 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796202c-a57a-4e65-8ec6-ed9d383b2893 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:40.410389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796202c-a57a-4e65-8ec6-ed9d383b2893 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:40.411794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4796202c-a57a-4e65-8ec6-ed9d383b2893 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-4796202c-a57a-4e65-8ec6-ed9d383b2893 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:37:40.423004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:40.424197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:40.425383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:40.425383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:40.425383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:37:40.425383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:37:40.425383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:37:40.425383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:37:40.425383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:37:40.425383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:37:40.425383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 19:37:40.425383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:37:40.425383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 594/1212] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:40.430747 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8c308-3df6-4158-9a91-18b19cbd18ed None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:40.525628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e8c308-3df6-4158-9a91-18b19cbd18ed None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:40.559440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f8504-c8b0-48a2-8584-d8fb0793b1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:40.560037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f8504-c8b0-48a2-8584-d8fb0793b1db None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:40.560316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f8504-c8b0-48a2-8584-d8fb0793b1db None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:40.560678 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f8504-c8b0-48a2-8584-d8fb0793b1db None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:40.561330 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f8504-c8b0-48a2-8584-d8fb0793b1db None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:40.561632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f8504-c8b0-48a2-8584-d8fb0793b1db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:40.563108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f8504-c8b0-48a2-8584-d8fb0793b1db None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:40.580167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 619/1213] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Sep 14 19:37:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:40.586135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796202c-a57a-4e65-8ec6-ed9d383b2893 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:40.684244 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796202c-a57a-4e65-8ec6-ed9d383b2893 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:40.716863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccca7d7-8fe4-47a2-8630-b8f7bec3253e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:40.717451 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccca7d7-8fe4-47a2-8630-b8f7bec3253e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:40.717724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccca7d7-8fe4-47a2-8630-b8f7bec3253e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:40.718017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccca7d7-8fe4-47a2-8630-b8f7bec3253e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:40.747398 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:40.783156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccca7d7-8fe4-47a2-8630-b8f7bec3253e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:40.784518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccca7d7-8fe4-47a2-8630-b8f7bec3253e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:40.878104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 595/1214] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:37:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:37:40.884646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f8504-c8b0-48a2-8584-d8fb0793b1db None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:40.903250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:40.980463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f8504-c8b0-48a2-8584-d8fb0793b1db None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:41.015261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966ce2f-c9c7-429e-bd41-9491d8035ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:41.015879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966ce2f-c9c7-429e-bd41-9491d8035ed3 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:41.016154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966ce2f-c9c7-429e-bd41-9491d8035ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:41.016422 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966ce2f-c9c7-429e-bd41-9491d8035ed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:41.026120 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966ce2f-c9c7-429e-bd41-9491d8035ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:41.027545 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966ce2f-c9c7-429e-bd41-9491d8035ed3 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-5966ce2f-c9c7-429e-bd41-9491d8035ed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:37:41.042342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:41.043429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:37:41.044531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:37:41.044531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:41.044531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:37:41.044531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 620/1215] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:41.049372 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccca7d7-8fe4-47a2-8630-b8f7bec3253e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:41.107997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:41.144167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccca7d7-8fe4-47a2-8630-b8f7bec3253e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:41.176407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8ae04-9dd1-42dd-8bd6-7dea79253a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:41.176997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8ae04-9dd1-42dd-8bd6-7dea79253a83 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:41.177270 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8ae04-9dd1-42dd-8bd6-7dea79253a83 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:41.177536 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a8ae04-9dd1-42dd-8bd6-7dea79253a83 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:41.178192 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8ae04-9dd1-42dd-8bd6-7dea79253a83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:41.179010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8ae04-9dd1-42dd-8bd6-7dea79253a83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:41.180355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a8ae04-9dd1-42dd-8bd6-7dea79253a83 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:41.191637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 596/1216] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:37:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:41.197452 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966ce2f-c9c7-429e-bd41-9491d8035ed3 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:41.264969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:41.298032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966ce2f-c9c7-429e-bd41-9491d8035ed3 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:41.336556 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade96d9-4e87-41d4-b428-2decc4eb65cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:41.337570 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade96d9-4e87-41d4-b428-2decc4eb65cf None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:41.338093 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade96d9-4e87-41d4-b428-2decc4eb65cf None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:41.338571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade96d9-4e87-41d4-b428-2decc4eb65cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:41.347671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade96d9-4e87-41d4-b428-2decc4eb65cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:41.350634 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade96d9-4e87-41d4-b428-2decc4eb65cf None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-dade96d9-4e87-41d4-b428-2decc4eb65cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:37:41.372078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:41.373094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:41.374020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:37:41.374020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:37:41.374020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:37:41.374020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:37:41.374020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:37:41.374020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:37:41.374020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:37:41.374020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:37:41.374020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 621/1217] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:37:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:41.380664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8ae04-9dd1-42dd-8bd6-7dea79253a83 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:41.493774 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a8ae04-9dd1-42dd-8bd6-7dea79253a83 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:41.518034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:41.529425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36042f-dbda-41ea-8d6a-9b4ec75b0795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:41.530049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36042f-dbda-41ea-8d6a-9b4ec75b0795 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:41.530325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36042f-dbda-41ea-8d6a-9b4ec75b0795 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:41.530592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36042f-dbda-41ea-8d6a-9b4ec75b0795 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:41.531264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36042f-dbda-41ea-8d6a-9b4ec75b0795 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:41.531564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36042f-dbda-41ea-8d6a-9b4ec75b0795 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:41.532979 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e36042f-dbda-41ea-8d6a-9b4ec75b0795 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:41.544901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 597/1218] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:37:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:41.551062 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade96d9-4e87-41d4-b428-2decc4eb65cf None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:41.648953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade96d9-4e87-41d4-b428-2decc4eb65cf None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:41.676734 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:41.682449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f94689f-f136-4c46-a5ae-73bf72240548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:41.683049 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94689f-f136-4c46-a5ae-73bf72240548 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:41.683334 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94689f-f136-4c46-a5ae-73bf72240548 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:41.683597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f94689f-f136-4c46-a5ae-73bf72240548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:41.684303 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94689f-f136-4c46-a5ae-73bf72240548 None admin] RBAC: Adding query filter params (role.id=928c1b8875334543ab431e4f7e2acee5, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=03bc280d0395479981b7c7b08c96001b) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:41.684609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94689f-f136-4c46-a5ae-73bf72240548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:41.686095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f94689f-f136-4c46-a5ae-73bf72240548 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:41.691531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 622/1219] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:37:41 2022] GET /identity/v3/role_assignments?user.id=03bc280d0395479981b7c7b08c96001b&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=928c1b8875334543ab431e4f7e2acee5 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:43.810319 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142d423-6c9d-48b2-8940-315fee515446 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:43.810872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142d423-6c9d-48b2-8940-315fee515446 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:43.810872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5142d423-6c9d-48b2-8940-315fee515446 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:43.812463 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 598/1220] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:37:43.817876 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370da0fa-6335-45c6-8630-48bd6b1787e7 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:43.818310 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370da0fa-6335-45c6-8630-48bd6b1787e7 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:43.818461 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370da0fa-6335-45c6-8630-48bd6b1787e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:43.836787 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-370da0fa-6335-45c6-8630-48bd6b1787e7 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:37:43.837991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-370da0fa-6335-45c6-8630-48bd6b1787e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:37:43.903406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 623/1221] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:37:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:37:44.179777 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142d423-6c9d-48b2-8940-315fee515446 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:44.278647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5142d423-6c9d-48b2-8940-315fee515446 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:44.310812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f56fe0-aeab-4e6c-849e-e9fcfc2ae55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:44.311427 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f56fe0-aeab-4e6c-849e-e9fcfc2ae55b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:44.311700 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f56fe0-aeab-4e6c-849e-e9fcfc2ae55b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:44.311964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f56fe0-aeab-4e6c-849e-e9fcfc2ae55b None admin] PATH_INFO: `/v3/roles/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:44.322444 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f56fe0-aeab-4e6c-849e-e9fcfc2ae55b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-41f56fe0-aeab-4e6c-849e-e9fcfc2ae55b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:37:44.327137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:44.328194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:44.329180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:44.329180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:44.329180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:44.329180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:44.329180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:37:44.329180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:37:44.329180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:37:44.329180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:37:44.329180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:37:44.329180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:37:44.329180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:37:44.329180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:37:44.329180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 599/1222] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:37:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:44.335134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370da0fa-6335-45c6-8630-48bd6b1787e7 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:44.429405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370da0fa-6335-45c6-8630-48bd6b1787e7 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:44.452689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:44.463020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1010b278-e707-4133-9ed7-30a7eeabeda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:44.463673 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010b278-e707-4133-9ed7-30a7eeabeda7 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:44.463954 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010b278-e707-4133-9ed7-30a7eeabeda7 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:44.464225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010b278-e707-4133-9ed7-30a7eeabeda7 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:44.464872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010b278-e707-4133-9ed7-30a7eeabeda7 None admin] RBAC: Adding query filter params (name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:44.465171 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010b278-e707-4133-9ed7-30a7eeabeda7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:44.466561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010b278-e707-4133-9ed7-30a7eeabeda7 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:44.474446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 624/1223] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:37:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:44.480220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f56fe0-aeab-4e6c-849e-e9fcfc2ae55b None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:44.523646 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:44.578547 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f56fe0-aeab-4e6c-849e-e9fcfc2ae55b None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:44.611286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b2d92-41f0-45d6-ad4a-e631c0ee703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:44.612418 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b2d92-41f0-45d6-ad4a-e631c0ee703d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:44.612418 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b2d92-41f0-45d6-ad4a-e631c0ee703d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:44.612418 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b2d92-41f0-45d6-ad4a-e631c0ee703d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:44.621752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b2d92-41f0-45d6-ad4a-e631c0ee703d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:44.623194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612b2d92-41f0-45d6-ad4a-e631c0ee703d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-612b2d92-41f0-45d6-ad4a-e631c0ee703d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:37:44.638265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:44.647977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:37:44.648966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:37:44.648966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:44.648966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:37:44.648966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 600/1224] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:44.649220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1010b278-e707-4133-9ed7-30a7eeabeda7 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:44.742212 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1010b278-e707-4133-9ed7-30a7eeabeda7 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:44.775859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067e5a-0d47-46d9-8b7d-e8ccc31e25e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:44.776482 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067e5a-0d47-46d9-8b7d-e8ccc31e25e6 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:44.776737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067e5a-0d47-46d9-8b7d-e8ccc31e25e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:44.777006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067e5a-0d47-46d9-8b7d-e8ccc31e25e6 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:44.777623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04067e5a-0d47-46d9-8b7d-e8ccc31e25e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:44.777949 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04067e5a-0d47-46d9-8b7d-e8ccc31e25e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:44.779298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04067e5a-0d47-46d9-8b7d-e8ccc31e25e6 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:44.790704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 625/1225] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:44.796373 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b2d92-41f0-45d6-ad4a-e631c0ee703d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:44.893469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:44.900545 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612b2d92-41f0-45d6-ad4a-e631c0ee703d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:44.931782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e205d-9e74-4861-8d16-5e6ac8ecdb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:44.932395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e205d-9e74-4861-8d16-5e6ac8ecdb59 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:44.932671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e205d-9e74-4861-8d16-5e6ac8ecdb59 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:44.932941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e205d-9e74-4861-8d16-5e6ac8ecdb59 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:44.939973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e205d-9e74-4861-8d16-5e6ac8ecdb59 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:44.941651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e205d-9e74-4861-8d16-5e6ac8ecdb59 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-280e205d-9e74-4861-8d16-5e6ac8ecdb59 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:37:44.952866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:44.954000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:44.954991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:44.954991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:44.954991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:37:44.954991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:37:44.954991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:37:44.954991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:37:44.954991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:37:44.954991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:37:44.954991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 19:37:44.954991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:37:44.954991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 601/1226] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:37:44.960426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067e5a-0d47-46d9-8b7d-e8ccc31e25e6 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:45.005143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:45.056721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067e5a-0d47-46d9-8b7d-e8ccc31e25e6 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:45.090364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b938b6f-533c-4116-93d9-b1f020a63b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:45.091071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b938b6f-533c-4116-93d9-b1f020a63b55 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:45.091343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b938b6f-533c-4116-93d9-b1f020a63b55 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:45.091607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b938b6f-533c-4116-93d9-b1f020a63b55 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:45.092353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b938b6f-533c-4116-93d9-b1f020a63b55 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:45.092650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b938b6f-533c-4116-93d9-b1f020a63b55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:45.094120 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b938b6f-533c-4116-93d9-b1f020a63b55 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:45.110534 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 626/1227] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Sep 14 19:37:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:37:45.116942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e205d-9e74-4861-8d16-5e6ac8ecdb59 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:45.214902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e205d-9e74-4861-8d16-5e6ac8ecdb59 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:45.246853 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07af6f-0455-4648-ab0e-4ad07f37e70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:45.247462 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07af6f-0455-4648-ab0e-4ad07f37e70c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:45.247734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07af6f-0455-4648-ab0e-4ad07f37e70c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:45.248000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f07af6f-0455-4648-ab0e-4ad07f37e70c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:45.258082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f07af6f-0455-4648-ab0e-4ad07f37e70c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:45.259432 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f07af6f-0455-4648-ab0e-4ad07f37e70c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-8f07af6f-0455-4648-ab0e-4ad07f37e70c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:37:45.273266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:45.276122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:37:45.277225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:37:45.277225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:45.277225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:37:45.277225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 602/1228] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:45.280155 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b938b6f-533c-4116-93d9-b1f020a63b55 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:45.374814 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b938b6f-533c-4116-93d9-b1f020a63b55 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:45.377078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:45.407370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cce203-99a7-498a-beb6-9e74ef43df25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:45.407970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cce203-99a7-498a-beb6-9e74ef43df25 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:45.408242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cce203-99a7-498a-beb6-9e74ef43df25 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:45.408508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cce203-99a7-498a-beb6-9e74ef43df25 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:45.409209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cce203-99a7-498a-beb6-9e74ef43df25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:45.409506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cce203-99a7-498a-beb6-9e74ef43df25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:45.410876 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cce203-99a7-498a-beb6-9e74ef43df25 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:45.422195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 627/1229] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:45.427787 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07af6f-0455-4648-ab0e-4ad07f37e70c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:45.436783 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:45.524689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f07af6f-0455-4648-ab0e-4ad07f37e70c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:45.557611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb4ba3-f7f1-4193-85b8-ebb256c35813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:45.558268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb4ba3-f7f1-4193-85b8-ebb256c35813 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:45.558544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb4ba3-f7f1-4193-85b8-ebb256c35813 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:45.558811 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb4ba3-f7f1-4193-85b8-ebb256c35813 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:45.565233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb4ba3-f7f1-4193-85b8-ebb256c35813 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:45.567111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb4ba3-f7f1-4193-85b8-ebb256c35813 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-aceb4ba3-f7f1-4193-85b8-ebb256c35813 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:37:45.578737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:45.579782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:45.580975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:37:45.580975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:37:45.580975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:37:45.580975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:37:45.580975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:37:45.580975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:37:45.580975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:37:45.580975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:37:45.580975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 603/1230] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:37:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:45.586201 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cce203-99a7-498a-beb6-9e74ef43df25 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:45.679692 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cce203-99a7-498a-beb6-9e74ef43df25 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:45.712727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23147c15-00c4-4dd1-a7ac-a214da6f4686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:45.713319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23147c15-00c4-4dd1-a7ac-a214da6f4686 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:45.713594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23147c15-00c4-4dd1-a7ac-a214da6f4686 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:45.713888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23147c15-00c4-4dd1-a7ac-a214da6f4686 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:45.714610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23147c15-00c4-4dd1-a7ac-a214da6f4686 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:45.714908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23147c15-00c4-4dd1-a7ac-a214da6f4686 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:45.716312 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23147c15-00c4-4dd1-a7ac-a214da6f4686 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:45.727762 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 628/1231] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:37:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:45.733856 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb4ba3-f7f1-4193-85b8-ebb256c35813 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:45.796842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:45.829652 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb4ba3-f7f1-4193-85b8-ebb256c35813 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:45.862284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac6d29-81f7-4d61-aec7-afa365dd85cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:45.862935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac6d29-81f7-4d61-aec7-afa365dd85cb None admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:45.863208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac6d29-81f7-4d61-aec7-afa365dd85cb None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:45.863477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac6d29-81f7-4d61-aec7-afa365dd85cb None admin] PATH_INFO: `/v3/projects/6bfc6c1c443849038af35577daf34088/users/03bc280d0395479981b7c7b08c96001b/roles/928c1b8875334543ab431e4f7e2acee5` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:45.878117 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac6d29-81f7-4d61-aec7-afa365dd85cb None admin] RBAC: Authorizing `identity:create_grant(project_id=6bfc6c1c443849038af35577daf34088, user_id=03bc280d0395479981b7c7b08c96001b, role_id=928c1b8875334543ab431e4f7e2acee5)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:45.879556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac6d29-81f7-4d61-aec7-afa365dd85cb None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:45.995769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 604/1232] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:37:45 2022] PUT /identity/v3/projects/6bfc6c1c443849038af35577daf34088/users/03bc280d0395479981b7c7b08c96001b/roles/928c1b8875334543ab431e4f7e2acee5 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:37:48.104976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0050b-6f56-4d60-b027-b48df46cfa64 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:48.105497 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0050b-6f56-4d60-b027-b48df46cfa64 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:48.105497 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0050b-6f56-4d60-b027-b48df46cfa64 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:48.106854 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 629/1233] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:37:48.112136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ffcaf-41a3-49dd-8b22-498d3b6db65f None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:48.112561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ffcaf-41a3-49dd-8b22-498d3b6db65f None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:48.112674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ffcaf-41a3-49dd-8b22-498d3b6db65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:48.131882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-df4ffcaf-41a3-49dd-8b22-498d3b6db65f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:37:48.132991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-df4ffcaf-41a3-49dd-8b22-498d3b6db65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:37:48.196961 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 605/1234] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:37:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:37:48.467408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0050b-6f56-4d60-b027-b48df46cfa64 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:48.492570 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:48.564352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0050b-6f56-4d60-b027-b48df46cfa64 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:48.598338 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb5f6f-afd5-4dae-bfc9-2b81df6cae55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:48.598954 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb5f6f-afd5-4dae-bfc9-2b81df6cae55 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:48.599229 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb5f6f-afd5-4dae-bfc9-2b81df6cae55 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:48.599496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb5f6f-afd5-4dae-bfc9-2b81df6cae55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:48.609167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb5f6f-afd5-4dae-bfc9-2b81df6cae55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-3fbb5f6f-afd5-4dae-bfc9-2b81df6cae55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:37:48.613823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:48.614906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:48.615818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:48.615818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:48.615818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:48.615818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:48.615818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:37:48.615818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:37:48.615818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:37:48.615818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:37:48.615818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:37:48.615818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:37:48.615818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 19:37:48.615818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:37:48.615818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 630/1235] 10.222.0.42 () {56 vars in 1102 bytes} [Wed Sep 14 19:37:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:48.620948 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ffcaf-41a3-49dd-8b22-498d3b6db65f None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:48.665948 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:48.716581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ffcaf-41a3-49dd-8b22-498d3b6db65f None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:48.748428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b456e2c-2126-4e1f-8a06-3a9a2bbb8b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:48.748968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b456e2c-2126-4e1f-8a06-3a9a2bbb8b67 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:48.749238 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b456e2c-2126-4e1f-8a06-3a9a2bbb8b67 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:48.749501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b456e2c-2126-4e1f-8a06-3a9a2bbb8b67 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:48.750176 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b456e2c-2126-4e1f-8a06-3a9a2bbb8b67 None admin] RBAC: Adding query filter params (name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:48.750474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b456e2c-2126-4e1f-8a06-3a9a2bbb8b67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:48.751818 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b456e2c-2126-4e1f-8a06-3a9a2bbb8b67 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:48.759797 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 606/1236] 10.222.0.42 () {56 vars in 1103 bytes} [Wed Sep 14 19:37:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:48.765682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb5f6f-afd5-4dae-bfc9-2b81df6cae55 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:48.848316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:48.863653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb5f6f-afd5-4dae-bfc9-2b81df6cae55 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:48.896811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce739848-cb9d-4437-a87c-7b196a5d8dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:48.897444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce739848-cb9d-4437-a87c-7b196a5d8dc0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:48.897718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce739848-cb9d-4437-a87c-7b196a5d8dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:48.898017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce739848-cb9d-4437-a87c-7b196a5d8dc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:48.908199 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce739848-cb9d-4437-a87c-7b196a5d8dc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:48.909721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce739848-cb9d-4437-a87c-7b196a5d8dc0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-ce739848-cb9d-4437-a87c-7b196a5d8dc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:37:48.923427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:48.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:37:48.925507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:37:48.925507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:48.925507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:37:48.925507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 631/1237] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:48.930924 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b456e2c-2126-4e1f-8a06-3a9a2bbb8b67 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:49.029864 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b456e2c-2126-4e1f-8a06-3a9a2bbb8b67 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:49.038224 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:49.061827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3df4c-ee77-4ee0-a5b7-10d1dc607e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:49.062436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3df4c-ee77-4ee0-a5b7-10d1dc607e8a None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:49.062708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3df4c-ee77-4ee0-a5b7-10d1dc607e8a None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:49.062970 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b3df4c-ee77-4ee0-a5b7-10d1dc607e8a None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:49.063590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3df4c-ee77-4ee0-a5b7-10d1dc607e8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:49.063884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3df4c-ee77-4ee0-a5b7-10d1dc607e8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:49.065254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b3df4c-ee77-4ee0-a5b7-10d1dc607e8a None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:49.077279 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 607/1238] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:49.082986 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce739848-cb9d-4437-a87c-7b196a5d8dc0 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:49.177648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce739848-cb9d-4437-a87c-7b196a5d8dc0 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:49.210290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97eaf3-c251-4760-8d5f-25b59933118b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:49.210898 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97eaf3-c251-4760-8d5f-25b59933118b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:49.211170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97eaf3-c251-4760-8d5f-25b59933118b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:49.211435 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a97eaf3-c251-4760-8d5f-25b59933118b None admin] PATH_INFO: `/v3/users/placement` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:49.214064 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:49.219019 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a97eaf3-c251-4760-8d5f-25b59933118b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:49.222439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a97eaf3-c251-4760-8d5f-25b59933118b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-0a97eaf3-c251-4760-8d5f-25b59933118b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:37:49.231348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:49.232428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:49.233530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:49.233530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:49.233530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:37:49.233530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:37:49.233530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:37:49.233530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:37:49.233530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:37:49.233530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:37:49.233530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 19:37:49.233530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:37:49.233530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 632/1239] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:49.238280 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3df4c-ee77-4ee0-a5b7-10d1dc607e8a None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:49.335569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b3df4c-ee77-4ee0-a5b7-10d1dc607e8a None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:49.368482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a668f68-77eb-4934-9e29-ac054d647439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:49.369067 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a668f68-77eb-4934-9e29-ac054d647439 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:49.369338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a668f68-77eb-4934-9e29-ac054d647439 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:49.369604 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a668f68-77eb-4934-9e29-ac054d647439 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:49.370340 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a668f68-77eb-4934-9e29-ac054d647439 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:49.370636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a668f68-77eb-4934-9e29-ac054d647439 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:49.372040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a668f68-77eb-4934-9e29-ac054d647439 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:49.388114 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 608/1240] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Sep 14 19:37:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:49.394137 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97eaf3-c251-4760-8d5f-25b59933118b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:49.489586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a97eaf3-c251-4760-8d5f-25b59933118b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:49.521990 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1c597-b437-4d9f-bd45-f6653312e360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:49.522582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1c597-b437-4d9f-bd45-f6653312e360 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:49.522852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1c597-b437-4d9f-bd45-f6653312e360 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:49.523117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b1c597-b437-4d9f-bd45-f6653312e360 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:49.581991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:49.588215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1c597-b437-4d9f-bd45-f6653312e360 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:49.589604 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b1c597-b437-4d9f-bd45-f6653312e360 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:49.682225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 633/1241] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:37:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:37:49.688136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a668f68-77eb-4934-9e29-ac054d647439 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:49.715772 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:49.785068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a668f68-77eb-4934-9e29-ac054d647439 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:49.817552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc7d3f-eec0-41de-9728-39c41875a5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:49.818219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc7d3f-eec0-41de-9728-39c41875a5d7 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:49.818439 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc7d3f-eec0-41de-9728-39c41875a5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:49.818730 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc7d3f-eec0-41de-9728-39c41875a5d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:49.828259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc7d3f-eec0-41de-9728-39c41875a5d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:49.829814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc7d3f-eec0-41de-9728-39c41875a5d7 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-c1dc7d3f-eec0-41de-9728-39c41875a5d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:37:49.844851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:49.845976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:37:49.847043 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:37:49.847043 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:49.847043 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:37:49.847043 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 609/1242] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:49.852218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1c597-b437-4d9f-bd45-f6653312e360 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:49.951893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:49.957399 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b1c597-b437-4d9f-bd45-f6653312e360 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:49.990587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0bf01-47ca-4fd4-ad6b-18a724d44f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:49.991178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0bf01-47ca-4fd4-ad6b-18a724d44f3f None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:49.991450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0bf01-47ca-4fd4-ad6b-18a724d44f3f None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:49.991714 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0bf01-47ca-4fd4-ad6b-18a724d44f3f None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:49.992335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0bf01-47ca-4fd4-ad6b-18a724d44f3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:49.992627 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0bf01-47ca-4fd4-ad6b-18a724d44f3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:49.994000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0bf01-47ca-4fd4-ad6b-18a724d44f3f None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:50.004558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 634/1243] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:50.010909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc7d3f-eec0-41de-9728-39c41875a5d7 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:50.087174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:50.107542 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc7d3f-eec0-41de-9728-39c41875a5d7 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:50.140441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4610c-7e3a-4cab-81ef-665c7df0136d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:50.141060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4610c-7e3a-4cab-81ef-665c7df0136d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:50.141334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4610c-7e3a-4cab-81ef-665c7df0136d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:50.141600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef4610c-7e3a-4cab-81ef-665c7df0136d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:50.148649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4610c-7e3a-4cab-81ef-665c7df0136d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:50.150085 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef4610c-7e3a-4cab-81ef-665c7df0136d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-8ef4610c-7e3a-4cab-81ef-665c7df0136d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:37:50.161341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:50.162386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:50.163334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:37:50.163334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:37:50.163334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:37:50.163334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:37:50.163334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:37:50.163334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:37:50.163334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:37:50.163334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:37:50.163334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 610/1244] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:37:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:50.168466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0bf01-47ca-4fd4-ad6b-18a724d44f3f None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:50.264996 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0bf01-47ca-4fd4-ad6b-18a724d44f3f None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:50.297532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab7e0c-3212-40ec-bc10-a3cb8cea9d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:50.298233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab7e0c-3212-40ec-bc10-a3cb8cea9d07 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:50.298508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab7e0c-3212-40ec-bc10-a3cb8cea9d07 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:50.298772 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab7e0c-3212-40ec-bc10-a3cb8cea9d07 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:50.299447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab7e0c-3212-40ec-bc10-a3cb8cea9d07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:50.299746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab7e0c-3212-40ec-bc10-a3cb8cea9d07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:50.301164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab7e0c-3212-40ec-bc10-a3cb8cea9d07 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:50.312855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 635/1245] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:37:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:50.320940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4610c-7e3a-4cab-81ef-665c7df0136d None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:50.422668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef4610c-7e3a-4cab-81ef-665c7df0136d None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:50.455458 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fccdb-1dc6-420b-b53c-ef1589e6ef78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:50.456047 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fccdb-1dc6-420b-b53c-ef1589e6ef78 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:50.456325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fccdb-1dc6-420b-b53c-ef1589e6ef78 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:50.456593 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fccdb-1dc6-420b-b53c-ef1589e6ef78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:50.457300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fccdb-1dc6-420b-b53c-ef1589e6ef78 None admin] RBAC: Adding query filter params (role.id=928c1b8875334543ab431e4f7e2acee5, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=03bc280d0395479981b7c7b08c96001b) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:50.457603 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fccdb-1dc6-420b-b53c-ef1589e6ef78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:50.459059 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34fccdb-1dc6-420b-b53c-ef1589e6ef78 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:50.464741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 611/1246] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:37:50 2022] GET /identity/v3/role_assignments?user.id=03bc280d0395479981b7c7b08c96001b&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=928c1b8875334543ab431e4f7e2acee5 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:52.651026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c6955-a1f7-4ea3-998c-35379c6e2bc1 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:52.651585 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c6955-a1f7-4ea3-998c-35379c6e2bc1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:52.651585 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c6955-a1f7-4ea3-998c-35379c6e2bc1 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:52.652911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 636/1247] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:37:52.658129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd6d46-c0d0-4c36-87be-050bd62fe7b4 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:52.658503 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd6d46-c0d0-4c36-87be-050bd62fe7b4 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:52.658677 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd6d46-c0d0-4c36-87be-050bd62fe7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:52.665592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:52.677569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e1cd6d46-c0d0-4c36-87be-050bd62fe7b4 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:37:52.678812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e1cd6d46-c0d0-4c36-87be-050bd62fe7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:37:52.745433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 612/1248] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:37:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:37:53.014489 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c6955-a1f7-4ea3-998c-35379c6e2bc1 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:53.034401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:53.115960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c6955-a1f7-4ea3-998c-35379c6e2bc1 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:53.149302 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3aebc-4325-4c87-a06e-1f1653e066f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:53.149956 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3aebc-4325-4c87-a06e-1f1653e066f9 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:53.150236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3aebc-4325-4c87-a06e-1f1653e066f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:53.150507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3aebc-4325-4c87-a06e-1f1653e066f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:53.160607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3aebc-4325-4c87-a06e-1f1653e066f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-52a3aebc-4325-4c87-a06e-1f1653e066f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:37:53.165324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:53.166522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:53.167572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:53.167572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:53.167572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:53.167572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:53.167572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:37:53.167572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:37:53.167572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:37:53.167572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:37:53.167572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:37:53.167572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:37:53.167572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:37:53.167572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:37:53.167572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 637/1249] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:37:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:53.172433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd6d46-c0d0-4c36-87be-050bd62fe7b4 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:53.267586 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd6d46-c0d0-4c36-87be-050bd62fe7b4 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:53.295493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:53.300010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f0bdd-642a-4a76-be48-6bdd5079cd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:53.300596 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0bdd-642a-4a76-be48-6bdd5079cd15 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:53.300867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0bdd-642a-4a76-be48-6bdd5079cd15 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:53.301132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f0bdd-642a-4a76-be48-6bdd5079cd15 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:53.301773 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f0bdd-642a-4a76-be48-6bdd5079cd15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:53.302094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f0bdd-642a-4a76-be48-6bdd5079cd15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:53.303433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325f0bdd-642a-4a76-be48-6bdd5079cd15 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:53.311066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 613/1250] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:37:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:53.317112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3aebc-4325-4c87-a06e-1f1653e066f9 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:53.401006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:53.422573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3aebc-4325-4c87-a06e-1f1653e066f9 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:53.455412 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a24841-b53c-4fc3-beb3-87c96e8f5dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:53.456041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a24841-b53c-4fc3-beb3-87c96e8f5dee None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:53.456318 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a24841-b53c-4fc3-beb3-87c96e8f5dee None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:53.456593 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a24841-b53c-4fc3-beb3-87c96e8f5dee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:53.467106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a24841-b53c-4fc3-beb3-87c96e8f5dee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:53.468544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a24841-b53c-4fc3-beb3-87c96e8f5dee None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-92a24841-b53c-4fc3-beb3-87c96e8f5dee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:37:53.482564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:53.483642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:37:53.484741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:37:53.484741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:53.484741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:37:53.484741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 638/1251] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:53.489449 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f0bdd-642a-4a76-be48-6bdd5079cd15 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:53.587720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f0bdd-642a-4a76-be48-6bdd5079cd15 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:53.620756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d26c4-0159-4383-840d-79eb2de44b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:53.621382 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d26c4-0159-4383-840d-79eb2de44b83 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:53.621655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d26c4-0159-4383-840d-79eb2de44b83 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:53.621983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d26c4-0159-4383-840d-79eb2de44b83 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:53.622620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d26c4-0159-4383-840d-79eb2de44b83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:53.622926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d26c4-0159-4383-840d-79eb2de44b83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:53.624400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d26c4-0159-4383-840d-79eb2de44b83 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:53.636667 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 614/1252] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:53.642444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a24841-b53c-4fc3-beb3-87c96e8f5dee None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:53.739123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a24841-b53c-4fc3-beb3-87c96e8f5dee None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:53.764755 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:53.772141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc6fcf-49c5-487d-94fc-15da6d6f2ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:53.772757 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc6fcf-49c5-487d-94fc-15da6d6f2ae8 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:53.773040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc6fcf-49c5-487d-94fc-15da6d6f2ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:53.773313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc6fcf-49c5-487d-94fc-15da6d6f2ae8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:53.780373 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc6fcf-49c5-487d-94fc-15da6d6f2ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:53.783690 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc6fcf-49c5-487d-94fc-15da6d6f2ae8 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-c0fc6fcf-49c5-487d-94fc-15da6d6f2ae8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:37:53.793039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:53.794202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:53.795659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:53.795659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:53.795659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:37:53.795659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:37:53.795659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:37:53.795659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:37:53.795659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:37:53.795659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:37:53.795659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 19:37:53.795659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:37:53.795659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 639/1253] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:53.799962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d26c4-0159-4383-840d-79eb2de44b83 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:53.853985 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:53.896872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d26c4-0159-4383-840d-79eb2de44b83 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:53.930016 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf562e-83e4-43fa-a215-15ca071ea2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:53.930615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf562e-83e4-43fa-a215-15ca071ea2bc None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:53.930886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf562e-83e4-43fa-a215-15ca071ea2bc None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:53.931154 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf562e-83e4-43fa-a215-15ca071ea2bc None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:53.931862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf562e-83e4-43fa-a215-15ca071ea2bc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:53.932161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf562e-83e4-43fa-a215-15ca071ea2bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:53.933570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cf562e-83e4-43fa-a215-15ca071ea2bc None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:53.950003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 615/1254] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Sep 14 19:37:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:53.956818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc6fcf-49c5-487d-94fc-15da6d6f2ae8 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:54.055118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc6fcf-49c5-487d-94fc-15da6d6f2ae8 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:54.086537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f64ad-007a-41be-98ea-41298b54797a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:54.087139 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f64ad-007a-41be-98ea-41298b54797a None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:54.087398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f64ad-007a-41be-98ea-41298b54797a None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:54.087662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f64ad-007a-41be-98ea-41298b54797a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:54.139540 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:54.151349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f64ad-007a-41be-98ea-41298b54797a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:54.152769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80f64ad-007a-41be-98ea-41298b54797a None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:54.244777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 640/1255] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:37:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:37:54.250657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf562e-83e4-43fa-a215-15ca071ea2bc None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:54.347257 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf562e-83e4-43fa-a215-15ca071ea2bc None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:54.360359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:54.380305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92bd37b-fac3-49b6-b4cb-4570b88c1063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:54.380922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bd37b-fac3-49b6-b4cb-4570b88c1063 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:54.381197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bd37b-fac3-49b6-b4cb-4570b88c1063 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:54.381464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92bd37b-fac3-49b6-b4cb-4570b88c1063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:54.390981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92bd37b-fac3-49b6-b4cb-4570b88c1063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:54.392349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92bd37b-fac3-49b6-b4cb-4570b88c1063 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-a92bd37b-fac3-49b6-b4cb-4570b88c1063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:37:54.407056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:54.408164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:37:54.409084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:37:54.409084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:54.409084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:37:54.409084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 616/1256] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:54.414558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f64ad-007a-41be-98ea-41298b54797a None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:54.496986 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:54.508618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f64ad-007a-41be-98ea-41298b54797a None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:54.541257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b1612-93f6-4f1b-a77f-3ff519999ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:54.541940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b1612-93f6-4f1b-a77f-3ff519999ded None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:54.542159 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b1612-93f6-4f1b-a77f-3ff519999ded None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:54.542429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b1612-93f6-4f1b-a77f-3ff519999ded None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:54.543054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b1612-93f6-4f1b-a77f-3ff519999ded None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:54.543402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b1612-93f6-4f1b-a77f-3ff519999ded None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:54.544761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b1612-93f6-4f1b-a77f-3ff519999ded None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:54.556237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 641/1257] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:54.562041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92bd37b-fac3-49b6-b4cb-4570b88c1063 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:54.657856 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92bd37b-fac3-49b6-b4cb-4570b88c1063 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:54.690212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2316400-e8ae-4b3c-917e-005cea0e8515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:54.691078 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2316400-e8ae-4b3c-917e-005cea0e8515 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:54.691353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2316400-e8ae-4b3c-917e-005cea0e8515 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:54.691621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2316400-e8ae-4b3c-917e-005cea0e8515 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:54.698041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2316400-e8ae-4b3c-917e-005cea0e8515 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:54.699467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2316400-e8ae-4b3c-917e-005cea0e8515 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-b2316400-e8ae-4b3c-917e-005cea0e8515 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:37:54.710409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:54.711374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:54.712284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:37:54.712284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:37:54.712284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:37:54.712284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:37:54.712284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:37:54.712284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:37:54.712284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:37:54.712284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:37:54.712284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 617/1258] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:37:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:54.717451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b1612-93f6-4f1b-a77f-3ff519999ded None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:54.814286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b1612-93f6-4f1b-a77f-3ff519999ded None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:54.846474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a8e14c-09c8-4bdf-b10e-7bb5d08ba222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:54.847068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8e14c-09c8-4bdf-b10e-7bb5d08ba222 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:54.847343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8e14c-09c8-4bdf-b10e-7bb5d08ba222 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:54.847616 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a8e14c-09c8-4bdf-b10e-7bb5d08ba222 None admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:54.848383 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8e14c-09c8-4bdf-b10e-7bb5d08ba222 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:54.848699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8e14c-09c8-4bdf-b10e-7bb5d08ba222 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:54.850657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a8e14c-09c8-4bdf-b10e-7bb5d08ba222 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:54.862193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 642/1259] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:37:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:54.868957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2316400-e8ae-4b3c-917e-005cea0e8515 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:54.894046 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:54.965938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2316400-e8ae-4b3c-917e-005cea0e8515 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:55.002904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0549dca2-b088-48e4-9753-07a96997cd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:55.015325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549dca2-b088-48e4-9753-07a96997cd4d None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:55.015325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549dca2-b088-48e4-9753-07a96997cd4d None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:55.015325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0549dca2-b088-48e4-9753-07a96997cd4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:55.015325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549dca2-b088-48e4-9753-07a96997cd4d None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=03bc280d0395479981b7c7b08c96001b) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:55.015325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549dca2-b088-48e4-9753-07a96997cd4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:55.015325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0549dca2-b088-48e4-9753-07a96997cd4d None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:55.016134 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 618/1260] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:37:54 2022] GET /identity/v3/role_assignments?user.id=03bc280d0395479981b7c7b08c96001b&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:57.097273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4af02e-5cfa-4c7b-82df-57221699e824 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:57.097879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4af02e-5cfa-4c7b-82df-57221699e824 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:57.097879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4af02e-5cfa-4c7b-82df-57221699e824 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:57.099219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 643/1261] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:37:57.104749 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd18292-f493-4710-893e-6c4a7fd81175 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:57.105194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd18292-f493-4710-893e-6c4a7fd81175 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:57.105295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd18292-f493-4710-893e-6c4a7fd81175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:57.123947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-4dd18292-f493-4710-893e-6c4a7fd81175 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:37:57.125056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4dd18292-f493-4710-893e-6c4a7fd81175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:37:57.189188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 619/1262] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:37:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:37:57.456788 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4af02e-5cfa-4c7b-82df-57221699e824 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:57.467613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:57.556214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4af02e-5cfa-4c7b-82df-57221699e824 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:57.589017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a831683-ba2a-46ff-b994-fca7426655ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:57.593342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831683-ba2a-46ff-b994-fca7426655ce None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:57.593342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831683-ba2a-46ff-b994-fca7426655ce None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:57.593342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831683-ba2a-46ff-b994-fca7426655ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:57.600602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a831683-ba2a-46ff-b994-fca7426655ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-3a831683-ba2a-46ff-b994-fca7426655ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:37:57.605340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:57.606584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:57.607486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:57.607486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:57.607486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:57.607486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:57.607486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:37:57.607486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:37:57.607486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:37:57.607486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:37:57.607486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:37:57.607486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:37:57.607486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:37:57.607486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:37:57.607486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 644/1263] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:37:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:57.612467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd18292-f493-4710-893e-6c4a7fd81175 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:57.630510 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:57.707286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd18292-f493-4710-893e-6c4a7fd81175 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:57.743142 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6580d0c-b563-430d-a2d9-7f7f79fb8b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:57.743752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6580d0c-b563-430d-a2d9-7f7f79fb8b47 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:57.744028 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6580d0c-b563-430d-a2d9-7f7f79fb8b47 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:57.744301 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6580d0c-b563-430d-a2d9-7f7f79fb8b47 None admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:57.744963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6580d0c-b563-430d-a2d9-7f7f79fb8b47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:57.745259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6580d0c-b563-430d-a2d9-7f7f79fb8b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:57.746653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6580d0c-b563-430d-a2d9-7f7f79fb8b47 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:57.754882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 620/1264] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:37:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:57.760667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a831683-ba2a-46ff-b994-fca7426655ce None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:57.826919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:57.860615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a831683-ba2a-46ff-b994-fca7426655ce None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:57.893581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d232f2-c7bd-4ff1-a0a6-75dfe0a2c3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:57.894227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d232f2-c7bd-4ff1-a0a6-75dfe0a2c3d5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:57.894504 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d232f2-c7bd-4ff1-a0a6-75dfe0a2c3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:57.894770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d232f2-c7bd-4ff1-a0a6-75dfe0a2c3d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:57.905497 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d232f2-c7bd-4ff1-a0a6-75dfe0a2c3d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:57.906953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d232f2-c7bd-4ff1-a0a6-75dfe0a2c3d5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-e3d232f2-c7bd-4ff1-a0a6-75dfe0a2c3d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:37:57.921089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:57.922279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:37:57.923574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:37:57.923574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:57.923574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:37:57.923574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 645/1265] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:57.928111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6580d0c-b563-430d-a2d9-7f7f79fb8b47 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:58.009908 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:58.028220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6580d0c-b563-430d-a2d9-7f7f79fb8b47 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:58.060658 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb8455-fc0c-4796-9666-ca814ddb65c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:58.071673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb8455-fc0c-4796-9666-ca814ddb65c8 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:58.071673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb8455-fc0c-4796-9666-ca814ddb65c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:58.071673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb8455-fc0c-4796-9666-ca814ddb65c8 None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:58.071673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb8455-fc0c-4796-9666-ca814ddb65c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:58.071673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb8455-fc0c-4796-9666-ca814ddb65c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:58.071673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb8455-fc0c-4796-9666-ca814ddb65c8 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:58.075473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 621/1266] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:58.081385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d232f2-c7bd-4ff1-a0a6-75dfe0a2c3d5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:58.177849 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d232f2-c7bd-4ff1-a0a6-75dfe0a2c3d5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:58.196573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:58.211483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bf6b4-3d7e-41db-9373-1cb3b673b57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:58.212133 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bf6b4-3d7e-41db-9373-1cb3b673b57b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:58.212408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bf6b4-3d7e-41db-9373-1cb3b673b57b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:58.212678 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7bf6b4-3d7e-41db-9373-1cb3b673b57b None admin] PATH_INFO: `/v3/users/placement` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:58.219953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7bf6b4-3d7e-41db-9373-1cb3b673b57b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:58.221406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7bf6b4-3d7e-41db-9373-1cb3b673b57b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-9e7bf6b4-3d7e-41db-9373-1cb3b673b57b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:37:58.232791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:58.240099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:58.241336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:58.241336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:58.241336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:37:58.241336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:37:58.241336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:37:58.241336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:37:58.241336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:37:58.241336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:37:58.241336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 19:37:58.241336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:37:58.241336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 646/1267] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:58.242133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb8455-fc0c-4796-9666-ca814ddb65c8 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:58.337708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb8455-fc0c-4796-9666-ca814ddb65c8 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:58.370025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242cd84-5425-4a55-ad45-9d3a75f92940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:58.370614 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242cd84-5425-4a55-ad45-9d3a75f92940 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:58.370886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242cd84-5425-4a55-ad45-9d3a75f92940 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:58.371150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242cd84-5425-4a55-ad45-9d3a75f92940 None admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:58.371858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242cd84-5425-4a55-ad45-9d3a75f92940 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:58.372158 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242cd84-5425-4a55-ad45-9d3a75f92940 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:58.373560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242cd84-5425-4a55-ad45-9d3a75f92940 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:58.382443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:58.390497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 622/1268] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Sep 14 19:37:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:58.396516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bf6b4-3d7e-41db-9373-1cb3b673b57b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:58.493315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7bf6b4-3d7e-41db-9373-1cb3b673b57b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:58.525544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05226281-bafe-4456-8f37-db1c03eaeb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:58.526181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05226281-bafe-4456-8f37-db1c03eaeb4d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:58.526456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05226281-bafe-4456-8f37-db1c03eaeb4d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:58.526731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05226281-bafe-4456-8f37-db1c03eaeb4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:58.538331 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05226281-bafe-4456-8f37-db1c03eaeb4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:58.544220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05226281-bafe-4456-8f37-db1c03eaeb4d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-05226281-bafe-4456-8f37-db1c03eaeb4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:37:58.553393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:58.554643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:37:58.555790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:37:58.555790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:37:58.555790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:37:58.555790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 647/1269] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:58.560294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242cd84-5425-4a55-ad45-9d3a75f92940 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:58.656521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242cd84-5425-4a55-ad45-9d3a75f92940 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:58.689752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a4ea3-f1af-448c-b2f1-3aeee7fbff8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:58.690382 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a4ea3-f1af-448c-b2f1-3aeee7fbff8c None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:58.690656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a4ea3-f1af-448c-b2f1-3aeee7fbff8c None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:58.691190 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13a4ea3-f1af-448c-b2f1-3aeee7fbff8c None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:58.691818 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a4ea3-f1af-448c-b2f1-3aeee7fbff8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:58.692116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a4ea3-f1af-448c-b2f1-3aeee7fbff8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:58.693558 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13a4ea3-f1af-448c-b2f1-3aeee7fbff8c None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:58.704883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 623/1270] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:58.710821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05226281-bafe-4456-8f37-db1c03eaeb4d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:58.741583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:58.805566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05226281-bafe-4456-8f37-db1c03eaeb4d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:58.837748 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22466fe-2c79-4441-adf3-490eea7606e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:58.838382 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22466fe-2c79-4441-adf3-490eea7606e4 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:58.838659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22466fe-2c79-4441-adf3-490eea7606e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:58.838926 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22466fe-2c79-4441-adf3-490eea7606e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:58.845227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22466fe-2c79-4441-adf3-490eea7606e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:58.846674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22466fe-2c79-4441-adf3-490eea7606e4 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-f22466fe-2c79-4441-adf3-490eea7606e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:37:58.857704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:37:58.858659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:37:58.859615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:37:58.859615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:37:58.859615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:37:58.859615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:37:58.859615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:37:58.859615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:37:58.859615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:37:58.859615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:37:58.859615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 648/1271] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:37:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:37:58.864804 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a4ea3-f1af-448c-b2f1-3aeee7fbff8c None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:58.921229 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:58.961894 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13a4ea3-f1af-448c-b2f1-3aeee7fbff8c None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:58.993907 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef9029-6e0a-4ade-959f-fc4b24343505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:58.994504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef9029-6e0a-4ade-959f-fc4b24343505 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:58.994776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef9029-6e0a-4ade-959f-fc4b24343505 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:58.995041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef9029-6e0a-4ade-959f-fc4b24343505 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:58.995711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef9029-6e0a-4ade-959f-fc4b24343505 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:37:58.996008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef9029-6e0a-4ade-959f-fc4b24343505 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:58.997402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef9029-6e0a-4ade-959f-fc4b24343505 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:59.009553 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 624/1272] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:37:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:37:59.015912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22466fe-2c79-4441-adf3-490eea7606e4 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:37:59.103503 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:37:59.111669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22466fe-2c79-4441-adf3-490eea7606e4 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:37:59.143874 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d01f87-a306-465e-bceb-e9103d747c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:37:59.144524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d01f87-a306-465e-bceb-e9103d747c3d None admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:37:59.144801 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d01f87-a306-465e-bceb-e9103d747c3d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:37:59.145120 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d01f87-a306-465e-bceb-e9103d747c3d None admin] PATH_INFO: `/v3/projects/6bfc6c1c443849038af35577daf34088/users/03bc280d0395479981b7c7b08c96001b/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:37:59.146995 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d01f87-a306-465e-bceb-e9103d747c3d None admin] RBAC: Authorizing `identity:create_grant(project_id=6bfc6c1c443849038af35577daf34088, user_id=03bc280d0395479981b7c7b08c96001b, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:37:59.148415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d01f87-a306-465e-bceb-e9103d747c3d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:37:59.248248 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 649/1273] 10.222.0.42 () {58 vars in 1439 bytes} [Wed Sep 14 19:37:59 2022] PUT /identity/v3/projects/6bfc6c1c443849038af35577daf34088/users/03bc280d0395479981b7c7b08c96001b/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:38:01.349594 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008b87b-9975-4674-9889-8ff0f75702b1 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:01.350691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008b87b-9975-4674-9889-8ff0f75702b1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:01.350691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008b87b-9975-4674-9889-8ff0f75702b1 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:01.351972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 625/1274] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:38:01.357385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42236d-ef8c-440e-b05c-4a429dee3463 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:01.357825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42236d-ef8c-440e-b05c-4a429dee3463 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:01.357949 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42236d-ef8c-440e-b05c-4a429dee3463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:01.376649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6e42236d-ef8c-440e-b05c-4a429dee3463 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:38:01.377924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6e42236d-ef8c-440e-b05c-4a429dee3463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:38:01.442768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 650/1275] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:38:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:38:01.716454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008b87b-9975-4674-9889-8ff0f75702b1 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:01.817109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008b87b-9975-4674-9889-8ff0f75702b1 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:01.835569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:01.851055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe46cf5-8f46-4e33-a729-e68902fc4ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:01.851706 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe46cf5-8f46-4e33-a729-e68902fc4ddf None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:01.851978 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe46cf5-8f46-4e33-a729-e68902fc4ddf None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:01.852238 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe46cf5-8f46-4e33-a729-e68902fc4ddf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:01.861894 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe46cf5-8f46-4e33-a729-e68902fc4ddf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-2fe46cf5-8f46-4e33-a729-e68902fc4ddf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 19:38:01.867104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:38:01.868246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:38:01.869377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:38:01.869377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:38:01.869377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:38:01.869377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:38:01.869377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 19:38:01.869377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 19:38:01.869377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 19:38:01.869377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 19:38:01.869377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 19:38:01.869377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 19:38:01.869377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 19:38:01.869377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:38:01.869377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 626/1276] 10.222.0.42 () {56 vars in 1096 bytes} [Wed Sep 14 19:38:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:38:01.874222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42236d-ef8c-440e-b05c-4a429dee3463 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:01.952090 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:01.970774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42236d-ef8c-440e-b05c-4a429dee3463 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:02.002158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baebb5-3ee6-4276-80dc-d862d499ef12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:02.002749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baebb5-3ee6-4276-80dc-d862d499ef12 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:02.003021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baebb5-3ee6-4276-80dc-d862d499ef12 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:02.003291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72baebb5-3ee6-4276-80dc-d862d499ef12 None admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:02.003927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baebb5-3ee6-4276-80dc-d862d499ef12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:38:02.004221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baebb5-3ee6-4276-80dc-d862d499ef12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:02.005614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72baebb5-3ee6-4276-80dc-d862d499ef12 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:02.013880 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 651/1277] 10.222.0.42 () {56 vars in 1099 bytes} [Wed Sep 14 19:38:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:38:02.019914 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe46cf5-8f46-4e33-a729-e68902fc4ddf None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:02.119293 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe46cf5-8f46-4e33-a729-e68902fc4ddf None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:02.151513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128a418-1e46-45fa-bc11-266cae989f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:02.152135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128a418-1e46-45fa-bc11-266cae989f77 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:02.157745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128a418-1e46-45fa-bc11-266cae989f77 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:02.157745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128a418-1e46-45fa-bc11-266cae989f77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:02.163534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128a418-1e46-45fa-bc11-266cae989f77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:02.164976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128a418-1e46-45fa-bc11-266cae989f77 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-d128a418-1e46-45fa-bc11-266cae989f77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:38:02.179362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:38:02.180471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:38:02.181588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:38:02.181588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:38:02.181588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:38:02.181588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 627/1278] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:38:02.186211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baebb5-3ee6-4276-80dc-d862d499ef12 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:02.286361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72baebb5-3ee6-4276-80dc-d862d499ef12 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:02.319443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d977d-620e-470b-a701-6fcf0abe8304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:02.320036 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d977d-620e-470b-a701-6fcf0abe8304 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:02.320335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d977d-620e-470b-a701-6fcf0abe8304 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:02.320602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d977d-620e-470b-a701-6fcf0abe8304 None admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:02.321223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d977d-620e-470b-a701-6fcf0abe8304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:38:02.321519 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d977d-620e-470b-a701-6fcf0abe8304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:02.323030 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d977d-620e-470b-a701-6fcf0abe8304 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:02.334265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 652/1279] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:38:02.340016 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128a418-1e46-45fa-bc11-266cae989f77 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:02.371394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:02.437851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128a418-1e46-45fa-bc11-266cae989f77 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:02.470703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20657aa9-7b86-4a27-a85a-78933438d2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:02.471305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20657aa9-7b86-4a27-a85a-78933438d2ba None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:02.471580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20657aa9-7b86-4a27-a85a-78933438d2ba None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:02.471852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20657aa9-7b86-4a27-a85a-78933438d2ba None admin] PATH_INFO: `/v3/users/placement` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:02.479503 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20657aa9-7b86-4a27-a85a-78933438d2ba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:02.480914 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20657aa9-7b86-4a27-a85a-78933438d2ba None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-20657aa9-7b86-4a27-a85a-78933438d2ba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 19:38:02.491566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:38:02.492644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:38:02.493621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:38:02.493621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:38:02.493621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 19:38:02.493621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 19:38:02.493621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 19:38:02.493621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 19:38:02.493621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 19:38:02.493621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 19:38:02.493621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 19:38:02.493621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:38:02.493621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 628/1280] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:38:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:38:02.498576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d977d-620e-470b-a701-6fcf0abe8304 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:02.502353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:02.595490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d977d-620e-470b-a701-6fcf0abe8304 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:02.629050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26992c20-621f-432b-b48d-ea3003ed07a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:02.629644 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26992c20-621f-432b-b48d-ea3003ed07a5 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:02.629948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26992c20-621f-432b-b48d-ea3003ed07a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:02.630218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26992c20-621f-432b-b48d-ea3003ed07a5 None admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:02.630918 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26992c20-621f-432b-b48d-ea3003ed07a5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:38:02.631215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26992c20-621f-432b-b48d-ea3003ed07a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:02.632617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26992c20-621f-432b-b48d-ea3003ed07a5 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:02.649656 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 653/1281] 10.222.0.42 () {56 vars in 1143 bytes} [Wed Sep 14 19:38:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:38:02.655626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20657aa9-7b86-4a27-a85a-78933438d2ba None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:02.744610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:02.754174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20657aa9-7b86-4a27-a85a-78933438d2ba None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:02.786746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc6925-32c7-4a9a-b22d-a0c17c5ebaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:02.787817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc6925-32c7-4a9a-b22d-a0c17c5ebaf3 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:02.788091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc6925-32c7-4a9a-b22d-a0c17c5ebaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:02.788357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc6925-32c7-4a9a-b22d-a0c17c5ebaf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:02.853344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc6925-32c7-4a9a-b22d-a0c17c5ebaf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:02.854761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc6925-32c7-4a9a-b22d-a0c17c5ebaf3 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:02.945766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:02.955601 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 629/1282] 10.222.0.42 () {58 vars in 1282 bytes} [Wed Sep 14 19:38:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 19:38:02.961437 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26992c20-621f-432b-b48d-ea3003ed07a5 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:03.028452 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:03.066135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26992c20-621f-432b-b48d-ea3003ed07a5 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:03.097580 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad35e3-12f5-495b-b3be-3d5ce968cb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:03.098270 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad35e3-12f5-495b-b3be-3d5ce968cb1b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:03.098543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad35e3-12f5-495b-b3be-3d5ce968cb1b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:03.098807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad35e3-12f5-495b-b3be-3d5ce968cb1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:03.108838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad35e3-12f5-495b-b3be-3d5ce968cb1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:03.110292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad35e3-12f5-495b-b3be-3d5ce968cb1b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-a7ad35e3-12f5-495b-b3be-3d5ce968cb1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 19:38:03.129325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:38:03.135138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 19:38:03.136319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 19:38:03.136319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 19:38:03.136319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:38:03.136319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 654/1283] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:38:03.136595 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc6925-32c7-4a9a-b22d-a0c17c5ebaf3 None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:03.233156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc6925-32c7-4a9a-b22d-a0c17c5ebaf3 None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:03.266457 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e42e46-4755-45c1-a3c2-e1ed4b528b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:03.267066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e42e46-4755-45c1-a3c2-e1ed4b528b0e None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:03.267339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e42e46-4755-45c1-a3c2-e1ed4b528b0e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:03.267608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e42e46-4755-45c1-a3c2-e1ed4b528b0e None admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:03.268237 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e42e46-4755-45c1-a3c2-e1ed4b528b0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:38:03.268534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e42e46-4755-45c1-a3c2-e1ed4b528b0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:03.269908 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e42e46-4755-45c1-a3c2-e1ed4b528b0e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:03.281384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 630/1284] 10.222.0.42 () {56 vars in 1109 bytes} [Wed Sep 14 19:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:38:03.287905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad35e3-12f5-495b-b3be-3d5ce968cb1b None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:03.384575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad35e3-12f5-495b-b3be-3d5ce968cb1b None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:03.398204 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:03.418215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615cf6b-1400-47ee-8d54-cb47381cfb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:03.418829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615cf6b-1400-47ee-8d54-cb47381cfb61 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:03.419102 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615cf6b-1400-47ee-8d54-cb47381cfb61 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:03.419367 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d615cf6b-1400-47ee-8d54-cb47381cfb61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:03.425704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615cf6b-1400-47ee-8d54-cb47381cfb61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:03.429785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d615cf6b-1400-47ee-8d54-cb47381cfb61 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application [None req-d615cf6b-1400-47ee-8d54-cb47381cfb61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 19:38:03.438225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application with Lock( Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:38:03.439255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:38:03.440387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 19:38:03.440387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 19:38:03.440387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 19:38:03.440387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 19:38:03.440387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 19:38:03.440387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 19:38:03.440387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 19:38:03.440387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: ERROR keystone.server.flask.application Sep 14 19:38:03.440387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 655/1285] 10.222.0.42 () {56 vars in 1111 bytes} [Wed Sep 14 19:38:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:38:03.445539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e42e46-4755-45c1-a3c2-e1ed4b528b0e None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:03.494871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:03.542517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e42e46-4755-45c1-a3c2-e1ed4b528b0e None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:03.574430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ced50-bf7a-4c26-883f-c7307a488ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:03.575040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ced50-bf7a-4c26-883f-c7307a488ad5 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:03.575313 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ced50-bf7a-4c26-883f-c7307a488ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:03.575578 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ced50-bf7a-4c26-883f-c7307a488ad5 None admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:03.576246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ced50-bf7a-4c26-883f-c7307a488ad5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:38:03.576544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ced50-bf7a-4c26-883f-c7307a488ad5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:03.578001 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ced50-bf7a-4c26-883f-c7307a488ad5 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:03.590033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 631/1286] 10.222.0.42 () {56 vars in 1148 bytes} [Wed Sep 14 19:38:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:38:03.596150 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615cf6b-1400-47ee-8d54-cb47381cfb61 None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:03.694578 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d615cf6b-1400-47ee-8d54-cb47381cfb61 None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:03.726537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf529be-8252-4ca9-bef3-4fa7dea26601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:03.727123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf529be-8252-4ca9-bef3-4fa7dea26601 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:03.727395 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf529be-8252-4ca9-bef3-4fa7dea26601 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:03.727658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf529be-8252-4ca9-bef3-4fa7dea26601 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:03.728351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf529be-8252-4ca9-bef3-4fa7dea26601 None admin] RBAC: Adding query filter params (role.id=70ef12c91aa140af8aaf02fe80527ebb, scope.project.id=6bfc6c1c443849038af35577daf34088, user.id=03bc280d0395479981b7c7b08c96001b) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:38:03.728648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf529be-8252-4ca9-bef3-4fa7dea26601 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:03.730082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf529be-8252-4ca9-bef3-4fa7dea26601 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:03.735546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 656/1287] 10.222.0.42 () {56 vars in 1374 bytes} [Wed Sep 14 19:38:03 2022] GET /identity/v3/role_assignments?user.id=03bc280d0395479981b7c7b08c96001b&scope.project.id=6bfc6c1c443849038af35577daf34088&role.id=70ef12c91aa140af8aaf02fe80527ebb => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:38:05.888032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb0917-3f24-4667-8856-01d5660a1e61 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:05.888443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb0917-3f24-4667-8856-01d5660a1e61 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:05.888584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb0917-3f24-4667-8856-01d5660a1e61 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:05.889935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 632/1288] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:38:05.895388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb1e6a-ffbc-47a0-a8c5-1e2f9e607018 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:05.895819 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb1e6a-ffbc-47a0-a8c5-1e2f9e607018 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:05.895912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb1e6a-ffbc-47a0-a8c5-1e2f9e607018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:05.914596 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-68bb1e6a-ffbc-47a0-a8c5-1e2f9e607018 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:38:05.915993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-68bb1e6a-ffbc-47a0-a8c5-1e2f9e607018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:38:05.944621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:05.980081 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 657/1289] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:38:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:38:06.250536 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb0917-3f24-4667-8856-01d5660a1e61 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:06.348784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badb0917-3f24-4667-8856-01d5660a1e61 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:06.358865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:06.381512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1198f-fb44-4648-9566-7e819e7a253b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:06.382161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1198f-fb44-4648-9566-7e819e7a253b None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:06.382433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1198f-fb44-4648-9566-7e819e7a253b None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:06.382699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1198f-fb44-4648-9566-7e819e7a253b None admin] PATH_INFO: `/v3/services/placement` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:06.383269 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d1198f-fb44-4648-9566-7e819e7a253b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:06.384613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d1198f-fb44-4648-9566-7e819e7a253b None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-12d1198f-fb44-4648-9566-7e819e7a253b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:38:06.393058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:38:06.394272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:38:06.395165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 633/1290] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 19:38:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 19:38:06.400086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb1e6a-ffbc-47a0-a8c5-1e2f9e607018 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:06.496227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bb1e6a-ffbc-47a0-a8c5-1e2f9e607018 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:06.528735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ae810-173f-4fef-b67b-d5bc96183bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:06.529275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ae810-173f-4fef-b67b-d5bc96183bd0 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:06.529551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ae810-173f-4fef-b67b-d5bc96183bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:06.529860 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ae810-173f-4fef-b67b-d5bc96183bd0 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:06.530464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ae810-173f-4fef-b67b-d5bc96183bd0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:38:06.530763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ae810-173f-4fef-b67b-d5bc96183bd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:06.532223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ae810-173f-4fef-b67b-d5bc96183bd0 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:06.536808 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 658/1291] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Sep 14 19:38:06 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:38:06.543214 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1198f-fb44-4648-9566-7e819e7a253b None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:06.641004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1198f-fb44-4648-9566-7e819e7a253b None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:06.674351 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064f7ec-59c1-408d-a654-9137326f9c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:06.674954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064f7ec-59c1-408d-a654-9137326f9c62 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:06.675230 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064f7ec-59c1-408d-a654-9137326f9c62 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:06.675500 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064f7ec-59c1-408d-a654-9137326f9c62 None admin] PATH_INFO: `/v3/services` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:06.676112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064f7ec-59c1-408d-a654-9137326f9c62 None admin] RBAC: Adding query filter params (type=placement) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:38:06.676413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064f7ec-59c1-408d-a654-9137326f9c62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:06.677765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064f7ec-59c1-408d-a654-9137326f9c62 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:06.682724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 634/1292] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Sep 14 19:38:06 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:38:08.810171 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2bf76-b1a2-4f3f-b5ef-351c76128a1d None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:08.810732 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2bf76-b1a2-4f3f-b5ef-351c76128a1d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:08.810732 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2bf76-b1a2-4f3f-b5ef-351c76128a1d None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:08.812011 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 659/1293] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:38:08.817714 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ffff1-0178-4b70-812d-596a3286f0aa None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:08.818201 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ffff1-0178-4b70-812d-596a3286f0aa None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:08.818301 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75ffff1-0178-4b70-812d-596a3286f0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:08.837240 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e75ffff1-0178-4b70-812d-596a3286f0aa None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:38:08.838463 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e75ffff1-0178-4b70-812d-596a3286f0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:38:08.867066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:08.904251 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 635/1294] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:38:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:38:09.172066 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2bf76-b1a2-4f3f-b5ef-351c76128a1d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:09.203709 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:09.270737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2bf76-b1a2-4f3f-b5ef-351c76128a1d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:09.302267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8856fb-59f3-4d63-aa7e-8b27be17abef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:09.303468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8856fb-59f3-4d63-aa7e-8b27be17abef None admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:09.303742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8856fb-59f3-4d63-aa7e-8b27be17abef None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:09.304008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8856fb-59f3-4d63-aa7e-8b27be17abef None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:09.304644 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8856fb-59f3-4d63-aa7e-8b27be17abef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:09.306006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8856fb-59f3-4d63-aa7e-8b27be17abef None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:09.390618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 660/1295] 10.222.0.42 () {60 vars in 1141 bytes} [Wed Sep 14 19:38:09 2022] POST /identity/v3/services => generated 246 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:38:11.501916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fe436-0efc-4941-8e77-ad384afed94c None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:11.502452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fe436-0efc-4941-8e77-ad384afed94c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:11.502452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fe436-0efc-4941-8e77-ad384afed94c None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:11.503782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 636/1296] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:38:11.508973 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55610a-06f5-4087-91be-d1238be1b762 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:11.509381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55610a-06f5-4087-91be-d1238be1b762 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:11.509573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55610a-06f5-4087-91be-d1238be1b762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:11.528083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-fe55610a-06f5-4087-91be-d1238be1b762 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:38:11.529250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fe55610a-06f5-4087-91be-d1238be1b762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:38:11.598977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 661/1297] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:38:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:38:11.862075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fe436-0efc-4941-8e77-ad384afed94c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:11.959921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fe436-0efc-4941-8e77-ad384afed94c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:11.992096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3aaca-389a-47dd-b885-b496362691bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:11.992705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3aaca-389a-47dd-b885-b496362691bc None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:11.992977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3aaca-389a-47dd-b885-b496362691bc None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:11.993239 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3aaca-389a-47dd-b885-b496362691bc None admin] PATH_INFO: `/v3/services/placement` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:11.993824 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3aaca-389a-47dd-b885-b496362691bc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:11.995169 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3aaca-389a-47dd-b885-b496362691bc None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-c7e3aaca-389a-47dd-b885-b496362691bc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:38:12.003570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:38:12.004815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:38:12.006513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 637/1298] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 19:38:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:38:12.011049 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55610a-06f5-4087-91be-d1238be1b762 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:12.029082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:12.104480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55610a-06f5-4087-91be-d1238be1b762 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:12.137047 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3c7ac-3e83-484b-8bb6-15f86c96037d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:12.137580 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3c7ac-3e83-484b-8bb6-15f86c96037d None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:12.137881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3c7ac-3e83-484b-8bb6-15f86c96037d None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:12.138152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3c7ac-3e83-484b-8bb6-15f86c96037d None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:12.138763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3c7ac-3e83-484b-8bb6-15f86c96037d None admin] RBAC: Adding query filter params (name=placement) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:38:12.139060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3c7ac-3e83-484b-8bb6-15f86c96037d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:12.140399 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3c7ac-3e83-484b-8bb6-15f86c96037d None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:12.145332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 662/1299] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Sep 14 19:38:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:38:12.151294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3aaca-389a-47dd-b885-b496362691bc None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:12.182457 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:12.247934 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3aaca-389a-47dd-b885-b496362691bc None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:12.281374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e4983d-f342-4219-b158-493404c09f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:12.281998 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4983d-f342-4219-b158-493404c09f41 None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:12.282274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4983d-f342-4219-b158-493404c09f41 None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:12.282541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e4983d-f342-4219-b158-493404c09f41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:12.283229 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4983d-f342-4219-b158-493404c09f41 None admin] RBAC: Adding query filter params (interface=public, service_id=9b51c35c157f4090b1e2687f939eff5d, region_id=RegionOne) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:38:12.283533 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4983d-f342-4219-b158-493404c09f41 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:12.284873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e4983d-f342-4219-b158-493404c09f41 None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:12.289789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 638/1300] 10.222.0.42 () {56 vars in 1251 bytes} [Wed Sep 14 19:38:12 2022] GET /identity/v3/endpoints?service_id=9b51c35c157f4090b1e2687f939eff5d&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:38:12.295488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3c7ac-3e83-484b-8bb6-15f86c96037d None admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:12.375531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:12.393836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3c7ac-3e83-484b-8bb6-15f86c96037d None admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:12.425278 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acf995e-57d9-4683-b953-7ff18b181b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:12.425894 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf995e-57d9-4683-b953-7ff18b181b31 None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:12.426168 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf995e-57d9-4683-b953-7ff18b181b31 None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:12.426804 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acf995e-57d9-4683-b953-7ff18b181b31 None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:12.426981 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf995e-57d9-4683-b953-7ff18b181b31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:12.428814 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acf995e-57d9-4683-b953-7ff18b181b31 None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:12.435280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 663/1301] 10.222.0.42 () {56 vars in 1087 bytes} [Wed Sep 14 19:38:12 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:38:14.550528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e231c9-0936-4599-b2b6-bf5e0e514c9b None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:14.551145 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e231c9-0936-4599-b2b6-bf5e0e514c9b None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:14.551145 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e231c9-0936-4599-b2b6-bf5e0e514c9b None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:14.552394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 639/1302] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:38:14.557858 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbb846-cd20-4e89-8115-171120fc84ec None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:14.558282 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbb846-cd20-4e89-8115-171120fc84ec None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:14.558432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbb846-cd20-4e89-8115-171120fc84ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:14.576457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-62dbb846-cd20-4e89-8115-171120fc84ec None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:38:14.577631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-62dbb846-cd20-4e89-8115-171120fc84ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:38:14.641633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 664/1303] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:38:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 19:38:14.911607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e231c9-0936-4599-b2b6-bf5e0e514c9b None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:15.003895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:15.011147 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e231c9-0936-4599-b2b6-bf5e0e514c9b None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:15.049780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2f4e7-88ae-460e-b896-fb63710f5bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:15.050426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2f4e7-88ae-460e-b896-fb63710f5bce None admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:15.050699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2f4e7-88ae-460e-b896-fb63710f5bce None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:15.050964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb2f4e7-88ae-460e-b896-fb63710f5bce None admin] PATH_INFO: `/v3/services/placement` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:15.051549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2f4e7-88ae-460e-b896-fb63710f5bce None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:15.052899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb2f4e7-88ae-460e-b896-fb63710f5bce None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application [None req-aeb2f4e7-88ae-460e-b896-fb63710f5bce None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 19:38:15.061345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 19:38:15.066036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application with Lock( Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self._enter() Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application return self.creator() Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application created_value = creator( Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: ERROR keystone.server.flask.application Sep 14 19:38:15.067384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 640/1304] 10.222.0.42 () {56 vars in 1117 bytes} [Wed Sep 14 19:38:14 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 19:38:15.069401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbb846-cd20-4e89-8115-171120fc84ec None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:15.143339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:15.164936 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbb846-cd20-4e89-8115-171120fc84ec None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:15.198113 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d6959-74bd-4bc6-b943-6d8fdd8ce52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:15.198710 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d6959-74bd-4bc6-b943-6d8fdd8ce52b None admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:15.198983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d6959-74bd-4bc6-b943-6d8fdd8ce52b None admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:15.199250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d6959-74bd-4bc6-b943-6d8fdd8ce52b None admin] PATH_INFO: `/v3/services` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:15.199857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d6959-74bd-4bc6-b943-6d8fdd8ce52b None admin] RBAC: Adding query filter params (name=placement) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:38:15.200154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d6959-74bd-4bc6-b943-6d8fdd8ce52b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:15.201544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d6959-74bd-4bc6-b943-6d8fdd8ce52b None admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:15.206500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 665/1305] 10.222.0.42 () {56 vars in 1116 bytes} [Wed Sep 14 19:38:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:38:15.212139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2f4e7-88ae-460e-b896-fb63710f5bce None admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:38:15.309518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb2f4e7-88ae-460e-b896-fb63710f5bce None admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:38:15.342162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f0499-6a45-45a0-af19-bd684beb688e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:38:15.342753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f0499-6a45-45a0-af19-bd684beb688e None admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:38:15.343028 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f0499-6a45-45a0-af19-bd684beb688e None admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:38:15.343351 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f0499-6a45-45a0-af19-bd684beb688e None admin] PATH_INFO: `/v3/endpoints` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:38:15.344014 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f0499-6a45-45a0-af19-bd684beb688e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:38:15.345357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f0499-6a45-45a0-af19-bd684beb688e None admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:38:15.464621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 641/1306] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:38:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:38:25.322157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=30 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:38:25.460216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:45:52.133697 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ffaa8-901a-46b0-aed9-40fd42c52379 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:45:52.134186 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ffaa8-901a-46b0-aed9-40fd42c52379 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:45:52.134325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ffaa8-901a-46b0-aed9-40fd42c52379 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:45:52.140956 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20809fc-dc0d-477c-98f4-fc8402c4191c None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:45:52.141361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20809fc-dc0d-477c-98f4-fc8402c4191c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:45:52.141500 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20809fc-dc0d-477c-98f4-fc8402c4191c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:45:52.144466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 666/1307] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:45:52 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:45:52.280944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-f20809fc-dc0d-477c-98f4-fc8402c4191c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:45:52.280944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-f20809fc-dc0d-477c-98f4-fc8402c4191c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:45:52.417887 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 642/1308] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:45:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:45:52.723494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d193a-49d0-484d-9412-2fec20f72994 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:45:52.723777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d193a-49d0-484d-9412-2fec20f72994 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:45:52.724308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d193a-49d0-484d-9412-2fec20f72994 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:45:52.725911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 667/1309] 10.222.0.42 () {54 vars in 965 bytes} [Wed Sep 14 19:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:45:52.731882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa46b1-2469-4afb-8839-a56f3b5e34ac None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:45:52.732167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa46b1-2469-4afb-8839-a56f3b5e34ac None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:45:52.732442 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aa46b1-2469-4afb-8839-a56f3b5e34ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:45:52.820626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-24aa46b1-2469-4afb-8839-a56f3b5e34ac None None] MFA Rules not processed for user `7103ff9544524393a8c7175998dcae3b`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:45:52.821905 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-24aa46b1-2469-4afb-8839-a56f3b5e34ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:45:52.846631 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:45:52.909846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 643/1310] 10.222.0.42 () {58 vars in 1062 bytes} [Wed Sep 14 19:45:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:45:52.919648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d193a-49d0-484d-9412-2fec20f72994 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:45:52.921530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-db3d193a-49d0-484d-9412-2fec20f72994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:45:53.049348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d193a-49d0-484d-9412-2fec20f72994 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:45:53.137274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6bf624-ca21-4171-b3f1-42eaade0d43b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:45:53.138599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bf624-ca21-4171-b3f1-42eaade0d43b service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:45:53.139022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bf624-ca21-4171-b3f1-42eaade0d43b service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:45:53.139437 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bf624-ca21-4171-b3f1-42eaade0d43b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:45:53.141235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2c6bf624-ca21-4171-b3f1-42eaade0d43b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:45:53.202756 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:45:53.264130 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bf624-ca21-4171-b3f1-42eaade0d43b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:45:53.265596 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6bf624-ca21-4171-b3f1-42eaade0d43b service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:45:53.287415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 668/1311] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:45:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:45:55.721693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df288-c30e-422f-bee3-a0acd14fcb62 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:45:55.722165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df288-c30e-422f-bee3-a0acd14fcb62 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:45:55.722288 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6df288-c30e-422f-bee3-a0acd14fcb62 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:45:55.723800 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 644/1312] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:45:55.729030 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ccb47-622d-4745-85d4-98339ccfd82e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:45:55.729966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ccb47-622d-4745-85d4-98339ccfd82e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:45:55.730160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ccb47-622d-4745-85d4-98339ccfd82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:45:55.813272 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-db1ccb47-622d-4745-85d4-98339ccfd82e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:45:55.814513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-db1ccb47-622d-4745-85d4-98339ccfd82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:45:55.916880 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 669/1313] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:45:56.198279 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df288-c30e-422f-bee3-a0acd14fcb62 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:45:56.199853 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bc6df288-c30e-422f-bee3-a0acd14fcb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:45:56.303153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6df288-c30e-422f-bee3-a0acd14fcb62 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:45:56.375286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:45:56.377930 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5acba5-f226-488e-8205-7178273d5f3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:45:56.378855 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5acba5-f226-488e-8205-7178273d5f3d service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:45:56.379136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5acba5-f226-488e-8205-7178273d5f3d service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:45:56.379409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5acba5-f226-488e-8205-7178273d5f3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:45:56.381085 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-fd5acba5-f226-488e-8205-7178273d5f3d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:45:56.502414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5acba5-f226-488e-8205-7178273d5f3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:45:56.503783 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5acba5-f226-488e-8205-7178273d5f3d service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:45:56.523460 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 645/1314] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:45:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:45:59.123555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4ac1b-4091-41d9-a012-bc0c54867f79 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:45:59.124085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4ac1b-4091-41d9-a012-bc0c54867f79 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:45:59.124085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4ac1b-4091-41d9-a012-bc0c54867f79 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:45:59.125341 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 670/1315] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:45:59.130599 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31147fe3-2b87-4d8b-8d24-8128454db129 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:45:59.131023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31147fe3-2b87-4d8b-8d24-8128454db129 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:45:59.131139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31147fe3-2b87-4d8b-8d24-8128454db129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:45:59.194567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-31147fe3-2b87-4d8b-8d24-8128454db129 None None] MFA Rules not processed for user `34554fb083404728bd4ff4fa2b4a8149`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:45:59.195751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-31147fe3-2b87-4d8b-8d24-8128454db129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:45:59.246662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:45:59.290323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 646/1316] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:45:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:45:59.571579 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4ac1b-4091-41d9-a012-bc0c54867f79 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:45:59.573646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-41c4ac1b-4091-41d9-a012-bc0c54867f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:45:59.583399 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:45:59.674407 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4ac1b-4091-41d9-a012-bc0c54867f79 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:45:59.746725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb50e1c-5629-4d02-9f40-a550d937f820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:45:59.747314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb50e1c-5629-4d02-9f40-a550d937f820 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:45:59.747586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb50e1c-5629-4d02-9f40-a550d937f820 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:45:59.747849 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb50e1c-5629-4d02-9f40-a550d937f820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:45:59.749398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4eb50e1c-5629-4d02-9f40-a550d937f820 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:45:59.810386 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:45:59.855496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb50e1c-5629-4d02-9f40-a550d937f820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:45:59.856974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb50e1c-5629-4d02-9f40-a550d937f820 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:45:59.875552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 671/1317] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:45:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:46:02.705045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbfd0f-7142-4a3e-8422-7c05159d74c9 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:02.705645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbfd0f-7142-4a3e-8422-7c05159d74c9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:02.705645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbfd0f-7142-4a3e-8422-7c05159d74c9 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:02.706939 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 647/1318] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:02.712077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92311f-dbd5-462e-8656-d6b03352a557 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:02.712485 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92311f-dbd5-462e-8656-d6b03352a557 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:02.712626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92311f-dbd5-462e-8656-d6b03352a557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:02.777580 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-be92311f-dbd5-462e-8656-d6b03352a557 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:02.778813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-be92311f-dbd5-462e-8656-d6b03352a557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:02.874808 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 672/1319] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:46:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:03.157687 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbfd0f-7142-4a3e-8422-7c05159d74c9 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:46:03.159283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c6dbfd0f-7142-4a3e-8422-7c05159d74c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:03.260016 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbfd0f-7142-4a3e-8422-7c05159d74c9 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:46:03.331641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51832ca-e70c-43b4-919c-3430bffb7768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:46:03.332235 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51832ca-e70c-43b4-919c-3430bffb7768 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:03.332510 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51832ca-e70c-43b4-919c-3430bffb7768 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:03.332782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51832ca-e70c-43b4-919c-3430bffb7768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:03.334386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d51832ca-e70c-43b4-919c-3430bffb7768 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:03.341655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:03.439973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51832ca-e70c-43b4-919c-3430bffb7768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:46:03.441318 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51832ca-e70c-43b4-919c-3430bffb7768 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:46:03.459292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 648/1320] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:46:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:46:06.522360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75e90c-74fb-41b8-a64c-df521f3634f4 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:06.522932 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75e90c-74fb-41b8-a64c-df521f3634f4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:06.522932 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad75e90c-74fb-41b8-a64c-df521f3634f4 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:06.524212 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 673/1321] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:06.529588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cad325-2228-4b94-a5fb-3b9e285969a7 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:06.530057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cad325-2228-4b94-a5fb-3b9e285969a7 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:06.530317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cad325-2228-4b94-a5fb-3b9e285969a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:06.595739 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-29cad325-2228-4b94-a5fb-3b9e285969a7 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:06.596941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-29cad325-2228-4b94-a5fb-3b9e285969a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:06.653729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:06.693513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 649/1322] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:46:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:06.978335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75e90c-74fb-41b8-a64c-df521f3634f4 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:46:06.979924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ad75e90c-74fb-41b8-a64c-df521f3634f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:06.998772 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:07.106407 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad75e90c-74fb-41b8-a64c-df521f3634f4 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:46:07.173124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f1d91-e511-4269-940c-fee5ad0b12ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:46:07.173731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f1d91-e511-4269-940c-fee5ad0b12ba service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:07.174036 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f1d91-e511-4269-940c-fee5ad0b12ba service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:07.174305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f1d91-e511-4269-940c-fee5ad0b12ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:07.175909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-804f1d91-e511-4269-940c-fee5ad0b12ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:07.246567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:07.284312 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f1d91-e511-4269-940c-fee5ad0b12ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:46:07.316260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f1d91-e511-4269-940c-fee5ad0b12ba service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:46:07.316260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 674/1323] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:46:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:46:09.509555 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd38d1-188e-468a-bc08-daae94167465 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:09.510656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd38d1-188e-468a-bc08-daae94167465 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:09.510656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd38d1-188e-468a-bc08-daae94167465 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:09.511982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 650/1324] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:09.517161 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3484a2e-ae74-4278-bdc6-531ec4c50a02 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:09.517565 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3484a2e-ae74-4278-bdc6-531ec4c50a02 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:09.517711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3484a2e-ae74-4278-bdc6-531ec4c50a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:09.589963 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b3484a2e-ae74-4278-bdc6-531ec4c50a02 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:09.591157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b3484a2e-ae74-4278-bdc6-531ec4c50a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:09.686960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 675/1325] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:46:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:09.982418 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd38d1-188e-468a-bc08-daae94167465 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:46:09.983993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-77dd38d1-188e-468a-bc08-daae94167465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:10.087611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd38d1-188e-468a-bc08-daae94167465 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:46:10.162913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ba6e0c-8814-4f88-934b-a31a49f9c3c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:46:10.163513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba6e0c-8814-4f88-934b-a31a49f9c3c3 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:10.163791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba6e0c-8814-4f88-934b-a31a49f9c3c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:10.164059 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ba6e0c-8814-4f88-934b-a31a49f9c3c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:10.165635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c3ba6e0c-8814-4f88-934b-a31a49f9c3c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:10.184058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:10.278562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba6e0c-8814-4f88-934b-a31a49f9c3c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:46:10.280030 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ba6e0c-8814-4f88-934b-a31a49f9c3c3 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:46:10.298758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 651/1326] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:46:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:46:12.514000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c289c6e-c04d-4a6a-b5bf-f453958aa7c1 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:12.514553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c289c6e-c04d-4a6a-b5bf-f453958aa7c1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:12.514553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c289c6e-c04d-4a6a-b5bf-f453958aa7c1 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:12.515903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 676/1327] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:12.521009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3263-2f93-4708-b9fa-15092f0b01d8 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:12.521388 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3263-2f93-4708-b9fa-15092f0b01d8 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:12.521556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc3263-2f93-4708-b9fa-15092f0b01d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:12.589408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-76bc3263-2f93-4708-b9fa-15092f0b01d8 None None] MFA Rules not processed for user `34554fb083404728bd4ff4fa2b4a8149`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:12.590567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-76bc3263-2f93-4708-b9fa-15092f0b01d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:12.655298 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:12.687151 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 652/1328] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:46:12 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:12.967679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c289c6e-c04d-4a6a-b5bf-f453958aa7c1 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:46:12.969206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5c289c6e-c04d-4a6a-b5bf-f453958aa7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:13.001848 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:13.076877 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c289c6e-c04d-4a6a-b5bf-f453958aa7c1 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:46:13.151487 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed2c8e-e67f-47a5-b6cb-63e75f7cd4de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:46:13.152093 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed2c8e-e67f-47a5-b6cb-63e75f7cd4de service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:13.152370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed2c8e-e67f-47a5-b6cb-63e75f7cd4de service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:13.152641 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed2c8e-e67f-47a5-b6cb-63e75f7cd4de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:13.154267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ceed2c8e-e67f-47a5-b6cb-63e75f7cd4de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:13.232422 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:13.263662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed2c8e-e67f-47a5-b6cb-63e75f7cd4de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:46:13.265088 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed2c8e-e67f-47a5-b6cb-63e75f7cd4de service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:46:13.283628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 677/1329] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:46:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:46:15.896112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef9f0b-47b6-42ad-bc0b-610ed00e66e0 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:15.896571 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef9f0b-47b6-42ad-bc0b-610ed00e66e0 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:15.896682 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef9f0b-47b6-42ad-bc0b-610ed00e66e0 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:15.898082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 653/1330] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:15.903385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ae263-0d4e-4f4e-9fb7-1cf6223a832d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:15.903799 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ae263-0d4e-4f4e-9fb7-1cf6223a832d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:15.903917 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ae263-0d4e-4f4e-9fb7-1cf6223a832d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:15.968945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bf0ae263-0d4e-4f4e-9fb7-1cf6223a832d None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:15.970189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bf0ae263-0d4e-4f4e-9fb7-1cf6223a832d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:16.064920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 678/1331] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:46:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:16.347130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef9f0b-47b6-42ad-bc0b-610ed00e66e0 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:46:16.348661 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-7cef9f0b-47b6-42ad-bc0b-610ed00e66e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:16.450495 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef9f0b-47b6-42ad-bc0b-610ed00e66e0 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:46:16.520521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fd766-c5c5-4b4b-845b-c219659eb419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:46:16.521113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fd766-c5c5-4b4b-845b-c219659eb419 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:16.521389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fd766-c5c5-4b4b-845b-c219659eb419 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:16.521653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fd766-c5c5-4b4b-845b-c219659eb419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:16.523254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c73fd766-c5c5-4b4b-845b-c219659eb419 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:16.549508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:16.629501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fd766-c5c5-4b4b-845b-c219659eb419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:46:16.630887 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fd766-c5c5-4b4b-845b-c219659eb419 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:46:16.649260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 654/1332] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:46:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:46:18.822310 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70ea1d-8e3e-4cff-a6fc-1a5d5270297b None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:18.822904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70ea1d-8e3e-4cff-a6fc-1a5d5270297b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:18.822904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a70ea1d-8e3e-4cff-a6fc-1a5d5270297b None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:18.824161 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 679/1333] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:18.829389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196270ad-c246-45c2-a130-81b3bd678e61 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:18.829783 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196270ad-c246-45c2-a130-81b3bd678e61 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:18.829975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196270ad-c246-45c2-a130-81b3bd678e61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:18.896390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-196270ad-c246-45c2-a130-81b3bd678e61 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:18.897620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-196270ad-c246-45c2-a130-81b3bd678e61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:18.971615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:18.994354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 655/1334] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:19.274001 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70ea1d-8e3e-4cff-a6fc-1a5d5270297b None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:46:19.276183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0a70ea1d-8e3e-4cff-a6fc-1a5d5270297b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:19.316573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:19.378635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a70ea1d-8e3e-4cff-a6fc-1a5d5270297b None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:46:19.450870 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d34e2-c008-4656-81fc-3abf48b9d2b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:46:19.451461 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d34e2-c008-4656-81fc-3abf48b9d2b2 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:19.451733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d34e2-c008-4656-81fc-3abf48b9d2b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:19.451998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d34e2-c008-4656-81fc-3abf48b9d2b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:19.453583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6f9d34e2-c008-4656-81fc-3abf48b9d2b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:19.534521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:19.559838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d34e2-c008-4656-81fc-3abf48b9d2b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:46:19.561687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d34e2-c008-4656-81fc-3abf48b9d2b2 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:46:19.580776 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 680/1335] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:46:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:46:22.339094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d7f49-6109-4ef7-8e20-1d2dab16cbc6 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:22.339634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d7f49-6109-4ef7-8e20-1d2dab16cbc6 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:22.339634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d7f49-6109-4ef7-8e20-1d2dab16cbc6 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:22.340959 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 656/1336] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:22.346112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d9fca-ef65-46b1-ba41-322c56e683c5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:22.346533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d9fca-ef65-46b1-ba41-322c56e683c5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:22.346642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d9fca-ef65-46b1-ba41-322c56e683c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:22.414280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-fb2d9fca-ef65-46b1-ba41-322c56e683c5 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:22.415492 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fb2d9fca-ef65-46b1-ba41-322c56e683c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:22.510925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 681/1337] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:46:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:22.792081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d7f49-6109-4ef7-8e20-1d2dab16cbc6 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:46:22.793636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-5b8d7f49-6109-4ef7-8e20-1d2dab16cbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:22.893501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d7f49-6109-4ef7-8e20-1d2dab16cbc6 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:46:22.964335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a6d5e-1801-47dd-98bb-d17c478ac577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:46:22.964920 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a6d5e-1801-47dd-98bb-d17c478ac577 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:22.965194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a6d5e-1801-47dd-98bb-d17c478ac577 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:22.965454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a6d5e-1801-47dd-98bb-d17c478ac577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:22.967044 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-612a6d5e-1801-47dd-98bb-d17c478ac577 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:23.001329 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:23.071170 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a6d5e-1801-47dd-98bb-d17c478ac577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:46:23.072601 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a6d5e-1801-47dd-98bb-d17c478ac577 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:46:23.090833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 657/1338] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:46:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:46:27.309365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddaa23-0b0f-4141-94b1-d21d6879e770 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:27.309866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddaa23-0b0f-4141-94b1-d21d6879e770 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:27.309968 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddaa23-0b0f-4141-94b1-d21d6879e770 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:27.311396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 682/1339] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:27.316512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea691d21-2d3e-4b3e-be4e-6547b2b9f1f5 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:27.317219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea691d21-2d3e-4b3e-be4e-6547b2b9f1f5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:27.317310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea691d21-2d3e-4b3e-be4e-6547b2b9f1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:27.388183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-ea691d21-2d3e-4b3e-be4e-6547b2b9f1f5 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:27.389483 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ea691d21-2d3e-4b3e-be4e-6547b2b9f1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:27.474394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:27.485733 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 658/1340] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:27.763986 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddaa23-0b0f-4141-94b1-d21d6879e770 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:46:27.765538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-98ddaa23-0b0f-4141-94b1-d21d6879e770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:27.818288 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:27.878679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddaa23-0b0f-4141-94b1-d21d6879e770 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:46:27.952693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8c6a8c-aefe-48c4-b253-e1ce292970b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:46:27.953218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c6a8c-aefe-48c4-b253-e1ce292970b0 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:27.953495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c6a8c-aefe-48c4-b253-e1ce292970b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:27.953762 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8c6a8c-aefe-48c4-b253-e1ce292970b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:27.955368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0d8c6a8c-aefe-48c4-b253-e1ce292970b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:28.049149 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:28.065706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c6a8c-aefe-48c4-b253-e1ce292970b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:46:28.067437 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8c6a8c-aefe-48c4-b253-e1ce292970b0 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:46:28.090008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 683/1341] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:46:31.217704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987e892-fe7c-4702-a376-76a5279fadca None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:31.218248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987e892-fe7c-4702-a376-76a5279fadca None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:31.218354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6987e892-fe7c-4702-a376-76a5279fadca None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:31.219711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 659/1342] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:31.225048 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842aaf1-63e5-4ce6-8a55-4dceec9967ca None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:31.225543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842aaf1-63e5-4ce6-8a55-4dceec9967ca None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:31.225642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b842aaf1-63e5-4ce6-8a55-4dceec9967ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:31.295458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b842aaf1-63e5-4ce6-8a55-4dceec9967ca None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:31.296652 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b842aaf1-63e5-4ce6-8a55-4dceec9967ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:31.391528 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 684/1343] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:46:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:31.674083 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987e892-fe7c-4702-a376-76a5279fadca None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:46:31.675660 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6987e892-fe7c-4702-a376-76a5279fadca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:31.783897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6987e892-fe7c-4702-a376-76a5279fadca None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:46:31.861183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d16e1-a61e-46f2-9d4f-5588af9ad7dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:46:31.861784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d16e1-a61e-46f2-9d4f-5588af9ad7dd service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:31.862086 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d16e1-a61e-46f2-9d4f-5588af9ad7dd service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:31.862355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d16e1-a61e-46f2-9d4f-5588af9ad7dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:31.864059 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a93d16e1-a61e-46f2-9d4f-5588af9ad7dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:31.907603 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:31.995395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d16e1-a61e-46f2-9d4f-5588af9ad7dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:46:31.996873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d16e1-a61e-46f2-9d4f-5588af9ad7dd service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:46:32.016624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 660/1344] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:46:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:46:36.292065 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15130c8-51ee-4c69-93a4-3505dcb45354 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:36.292530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15130c8-51ee-4c69-93a4-3505dcb45354 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:36.292649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15130c8-51ee-4c69-93a4-3505dcb45354 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:36.293973 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 685/1345] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:46:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:46:36.299531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcfb21-1a67-4951-b0b4-255d0e0d7158 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:36.300041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcfb21-1a67-4951-b0b4-255d0e0d7158 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:36.300299 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcfb21-1a67-4951-b0b4-255d0e0d7158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:36.367752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-8cbcfb21-1a67-4951-b0b4-255d0e0d7158 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:36.369552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8cbcfb21-1a67-4951-b0b4-255d0e0d7158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:36.465857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:36.468062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 661/1346] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:36.749465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15130c8-51ee-4c69-93a4-3505dcb45354 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:46:36.751369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c15130c8-51ee-4c69-93a4-3505dcb45354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:36.807701 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:36.852329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15130c8-51ee-4c69-93a4-3505dcb45354 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:46:36.928312 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1db9d-38b7-42e3-8b83-b52b887d40ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:46:36.928907 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1db9d-38b7-42e3-8b83-b52b887d40ee service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:36.929180 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1db9d-38b7-42e3-8b83-b52b887d40ee service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:36.929442 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1db9d-38b7-42e3-8b83-b52b887d40ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:36.931049 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-07d1db9d-38b7-42e3-8b83-b52b887d40ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:37.034657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:37.037984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1db9d-38b7-42e3-8b83-b52b887d40ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:46:37.039398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1db9d-38b7-42e3-8b83-b52b887d40ee service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:46:37.057063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 686/1347] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:46:41.907182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1092f8-5043-4507-8650-4aeb8092cc18 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:41.908298 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1092f8-5043-4507-8650-4aeb8092cc18 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:41.909080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1092f8-5043-4507-8650-4aeb8092cc18 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:41.910997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 662/1348] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 19:46:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:41.916507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b856b-4546-4d2f-bc18-e19a599dbe81 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:41.917502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b856b-4546-4d2f-bc18-e19a599dbe81 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:41.917971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b856b-4546-4d2f-bc18-e19a599dbe81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:42.024469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-782b856b-4546-4d2f-bc18-e19a599dbe81 None None] MFA Rules not processed for user `03bc280d0395479981b7c7b08c96001b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:42.026198 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-782b856b-4546-4d2f-bc18-e19a599dbe81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:42.172092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 687/1349] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 19:46:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:46.314604 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53a62e-f6b3-4e01-bfd5-63c343e7e7e1 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:46.315182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53a62e-f6b3-4e01-bfd5-63c343e7e7e1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:46.315182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53a62e-f6b3-4e01-bfd5-63c343e7e7e1 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:46.316934 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 663/1350] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 19:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:46.321674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df2b4d-65e4-42a8-a6e5-6d2594c5cdca None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:46.322180 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df2b4d-65e4-42a8-a6e5-6d2594c5cdca None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:46.322470 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df2b4d-65e4-42a8-a6e5-6d2594c5cdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:46.379389 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:46.405578 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-97df2b4d-65e4-42a8-a6e5-6d2594c5cdca None None] MFA Rules not processed for user `03bc280d0395479981b7c7b08c96001b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:46.407052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-97df2b4d-65e4-42a8-a6e5-6d2594c5cdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:46.519511 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 688/1351] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 19:46:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:47.939701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05182c50-06c6-495e-979c-f642e89fc9db None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:47.940271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05182c50-06c6-495e-979c-f642e89fc9db None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:47.940271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05182c50-06c6-495e-979c-f642e89fc9db None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:47.941545 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 664/1352] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 19:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:47.946850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17c295-b9e0-4eeb-bb6d-5249a3d361b3 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:47.947277 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17c295-b9e0-4eeb-bb6d-5249a3d361b3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:47.947507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da17c295-b9e0-4eeb-bb6d-5249a3d361b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:48.024642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-da17c295-b9e0-4eeb-bb6d-5249a3d361b3 None None] MFA Rules not processed for user `03bc280d0395479981b7c7b08c96001b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:48.026216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-da17c295-b9e0-4eeb-bb6d-5249a3d361b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:48.069527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:48.137214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 689/1353] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 19:46:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:51.598599 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a476b-73e9-454d-8cdd-9a4048456972 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:51.599197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a476b-73e9-454d-8cdd-9a4048456972 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:51.599197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993a476b-73e9-454d-8cdd-9a4048456972 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:51.600440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 665/1354] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:46:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:51.605729 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da889d-06a6-431c-87c0-b3a64edd8765 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:51.606227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da889d-06a6-431c-87c0-b3a64edd8765 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:51.606341 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23da889d-06a6-431c-87c0-b3a64edd8765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:51.675328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-23da889d-06a6-431c-87c0-b3a64edd8765 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:51.676692 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-23da889d-06a6-431c-87c0-b3a64edd8765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:51.784868 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 690/1355] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:46:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:52.094330 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145af5f-0aa2-43a3-be83-a3accc882ed1 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:52.094889 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145af5f-0aa2-43a3-be83-a3accc882ed1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:52.095360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145af5f-0aa2-43a3-be83-a3accc882ed1 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:52.096946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 666/1356] 10.222.0.42 () {54 vars in 962 bytes} [Wed Sep 14 19:46:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:52.104145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe176c-de8e-44a7-ad01-0dfd7cac2f7f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:52.104439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe176c-de8e-44a7-ad01-0dfd7cac2f7f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:52.104717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbe176c-de8e-44a7-ad01-0dfd7cac2f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:52.128198 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:52.195273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7dbe176c-de8e-44a7-ad01-0dfd7cac2f7f None None] MFA Rules not processed for user `2a4284b59f804dc59674d621d87634bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:52.197011 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7dbe176c-de8e-44a7-ad01-0dfd7cac2f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:52.321833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 691/1357] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Sep 14 19:46:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:46:52.329316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145af5f-0aa2-43a3-be83-a3accc882ed1 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:46:52.330874 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d145af5f-0aa2-43a3-be83-a3accc882ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:52.505221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145af5f-0aa2-43a3-be83-a3accc882ed1 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:46:52.592720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:52.641190 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07dcfc0-06c5-4bf2-a1db-8bf33a42e142 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:46:52.642596 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dcfc0-06c5-4bf2-a1db-8bf33a42e142 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:52.643093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dcfc0-06c5-4bf2-a1db-8bf33a42e142 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:52.643577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07dcfc0-06c5-4bf2-a1db-8bf33a42e142 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:52.645400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b07dcfc0-06c5-4bf2-a1db-8bf33a42e142 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:52.745090 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07dcfc0-06c5-4bf2-a1db-8bf33a42e142 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:46:52.747202 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07dcfc0-06c5-4bf2-a1db-8bf33a42e142 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:46:52.767548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 667/1358] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:46:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:46:55.136101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d866b4d-4263-4306-90c9-8dabffcb8a88 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:55.136768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d866b4d-4263-4306-90c9-8dabffcb8a88 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:55.136768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d866b4d-4263-4306-90c9-8dabffcb8a88 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:55.143683 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7ea4f-8ed2-494e-a555-083dfd8f4128 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:55.144246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7ea4f-8ed2-494e-a555-083dfd8f4128 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:55.144246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7ea4f-8ed2-494e-a555-083dfd8f4128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:55.152095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 692/1359] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:46:55 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:55.214519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e9f7ea4f-8ed2-494e-a555-083dfd8f4128 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:55.215808 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e9f7ea4f-8ed2-494e-a555-083dfd8f4128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:55.257707 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:55.325672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 668/1360] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:55.614482 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d866b4d-4263-4306-90c9-8dabffcb8a88 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:46:55.615991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2d866b4d-4263-4306-90c9-8dabffcb8a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:55.683368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:55.784112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d866b4d-4263-4306-90c9-8dabffcb8a88 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:46:55.924269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d317611-e12c-4e9b-a17b-4420274fd04f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:46:55.924870 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d317611-e12c-4e9b-a17b-4420274fd04f service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:55.925150 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d317611-e12c-4e9b-a17b-4420274fd04f service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:55.925412 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d317611-e12c-4e9b-a17b-4420274fd04f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:55.927019 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1d317611-e12c-4e9b-a17b-4420274fd04f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:55.952634 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:56.023218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d317611-e12c-4e9b-a17b-4420274fd04f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:46:56.025354 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d317611-e12c-4e9b-a17b-4420274fd04f service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:46:56.044311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 693/1361] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:46:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:46:58.688724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457774b4-dbd0-4a3c-a96c-30983d27a35a None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:58.689267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457774b4-dbd0-4a3c-a96c-30983d27a35a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:58.689267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457774b4-dbd0-4a3c-a96c-30983d27a35a None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:58.690701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 669/1362] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:46:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:58.696019 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff75cd-04bb-4a19-b7fc-a03062511ab9 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:58.696452 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff75cd-04bb-4a19-b7fc-a03062511ab9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:58.696654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff75cd-04bb-4a19-b7fc-a03062511ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:58.765896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-55ff75cd-04bb-4a19-b7fc-a03062511ab9 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:58.767173 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-55ff75cd-04bb-4a19-b7fc-a03062511ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:58.873285 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:58.875980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 694/1363] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:59.171202 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439df656-2924-4667-842f-15fe7a704390 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:59.171743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439df656-2924-4667-842f-15fe7a704390 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:59.172217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439df656-2924-4667-842f-15fe7a704390 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:59.174178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 670/1364] 10.222.0.42 () {54 vars in 962 bytes} [Wed Sep 14 19:46:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:46:59.179055 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf7d36-3e62-4e0d-9f24-5ef76a461671 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:59.185342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf7d36-3e62-4e0d-9f24-5ef76a461671 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:59.185342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf7d36-3e62-4e0d-9f24-5ef76a461671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:59.257235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e4cf7d36-3e62-4e0d-9f24-5ef76a461671 None None] MFA Rules not processed for user `2a4284b59f804dc59674d621d87634bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:59.259054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e4cf7d36-3e62-4e0d-9f24-5ef76a461671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:59.377068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 695/1365] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Sep 14 19:46:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:46:59.386208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439df656-2924-4667-842f-15fe7a704390 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:46:59.387821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-439df656-2924-4667-842f-15fe7a704390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:59.540257 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439df656-2924-4667-842f-15fe7a704390 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:46:59.561672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:59.659785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc632f2-38a1-428a-ab9d-d2047405414b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:46:59.660412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc632f2-38a1-428a-ab9d-d2047405414b service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:59.660686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc632f2-38a1-428a-ab9d-d2047405414b service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:59.660956 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc632f2-38a1-428a-ab9d-d2047405414b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:59.662735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bcc632f2-38a1-428a-ab9d-d2047405414b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:59.760171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc632f2-38a1-428a-ab9d-d2047405414b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:46:59.762412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc632f2-38a1-428a-ab9d-d2047405414b service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:46:59.772464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4ae4a-72d1-41d8-955b-f4341681208e None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:59.772952 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4ae4a-72d1-41d8-955b-f4341681208e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:59.773050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea4ae4a-72d1-41d8-955b-f4341681208e None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:59.775084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 696/1366] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:46:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:46:59.779859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66422a80-d09f-4910-8a42-c9fd979008fd None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:46:59.780141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66422a80-d09f-4910-8a42-c9fd979008fd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:46:59.780662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66422a80-d09f-4910-8a42-c9fd979008fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:46:59.783551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 671/1367] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:46:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:46:59.842849 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:46:59.852467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-66422a80-d09f-4910-8a42-c9fd979008fd None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:46:59.853700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-66422a80-d09f-4910-8a42-c9fd979008fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:46:59.969452 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 697/1368] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:46:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:47:00.306239 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874ea45-c5f7-428a-887c-cf4672bf5ccc None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:00.306524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874ea45-c5f7-428a-887c-cf4672bf5ccc None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:00.306793 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874ea45-c5f7-428a-887c-cf4672bf5ccc None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:00.308390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 672/1369] 10.222.0.42 () {54 vars in 964 bytes} [Wed Sep 14 19:47:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:00.313106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221c986-4d10-46ae-9a28-df93cbd17655 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:00.313394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221c986-4d10-46ae-9a28-df93cbd17655 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:00.313667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221c986-4d10-46ae-9a28-df93cbd17655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:00.392490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e221c986-4d10-46ae-9a28-df93cbd17655 None None] MFA Rules not processed for user `8a2c9eec08b142138569043e3b34c6a8`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:00.393694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e221c986-4d10-46ae-9a28-df93cbd17655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:00.466265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:00.473841 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 698/1370] 10.222.0.42 () {58 vars in 1061 bytes} [Wed Sep 14 19:47:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:00.481909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874ea45-c5f7-428a-887c-cf4672bf5ccc None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:00.483436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4874ea45-c5f7-428a-887c-cf4672bf5ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:00.516834 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:00.600639 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874ea45-c5f7-428a-887c-cf4672bf5ccc None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:00.682540 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242145a-770e-4586-91f2-15cd8ced884a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:00.683149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242145a-770e-4586-91f2-15cd8ced884a service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:00.683422 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242145a-770e-4586-91f2-15cd8ced884a service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:00.683704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242145a-770e-4586-91f2-15cd8ced884a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:00.689002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-7242145a-770e-4586-91f2-15cd8ced884a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:00.770067 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:00.811284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242145a-770e-4586-91f2-15cd8ced884a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:00.812439 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7242145a-770e-4586-91f2-15cd8ced884a service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:00.841925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 673/1371] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:02.220097 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffce4c5-8f05-490a-8749-db44d7ab9b0a None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:02.220551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffce4c5-8f05-490a-8749-db44d7ab9b0a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:02.220646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffce4c5-8f05-490a-8749-db44d7ab9b0a None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:02.222040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 699/1372] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:02.228642 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1e6fb-4ae0-4c94-83db-1b5dbb6c26c0 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:02.229124 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1e6fb-4ae0-4c94-83db-1b5dbb6c26c0 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:02.229221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e1e6fb-4ae0-4c94-83db-1b5dbb6c26c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:02.294866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-78e1e6fb-4ae0-4c94-83db-1b5dbb6c26c0 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:02.296086 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-78e1e6fb-4ae0-4c94-83db-1b5dbb6c26c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:02.391991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 674/1373] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:47:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:02.692246 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffce4c5-8f05-490a-8749-db44d7ab9b0a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:02.693858 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1ffce4c5-8f05-490a-8749-db44d7ab9b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:02.844002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffce4c5-8f05-490a-8749-db44d7ab9b0a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:02.918775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:02.970995 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ac009-d25a-48e0-a973-26726325ce4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:02.971674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ac009-d25a-48e0-a973-26726325ce4a service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:02.971876 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ac009-d25a-48e0-a973-26726325ce4a service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:02.972145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ac009-d25a-48e0-a973-26726325ce4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:02.973766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-758ac009-d25a-48e0-a973-26726325ce4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:03.069095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ac009-d25a-48e0-a973-26726325ce4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:03.071321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ac009-d25a-48e0-a973-26726325ce4a service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:03.089334 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 700/1374] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:05.425754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb30df-044b-41ce-b259-26d18d0f13e5 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:05.426358 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb30df-044b-41ce-b259-26d18d0f13e5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:05.426358 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcb30df-044b-41ce-b259-26d18d0f13e5 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:05.427800 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 675/1375] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:05.432841 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b72935-ce68-44f5-88fc-51a54a24ed93 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:05.433234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b72935-ce68-44f5-88fc-51a54a24ed93 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:05.433377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b72935-ce68-44f5-88fc-51a54a24ed93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:05.500862 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-21b72935-ce68-44f5-88fc-51a54a24ed93 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:05.502071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-21b72935-ce68-44f5-88fc-51a54a24ed93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:05.532167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:05.603483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 701/1376] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:47:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:05.890634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb30df-044b-41ce-b259-26d18d0f13e5 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:05.892126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0fcb30df-044b-41ce-b259-26d18d0f13e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:05.929404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:06.044566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcb30df-044b-41ce-b259-26d18d0f13e5 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:06.157101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:06.163437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8cfd9-efe4-4cfa-97e0-23eb6b0a6cf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:06.165373 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8cfd9-efe4-4cfa-97e0-23eb6b0a6cf4 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:06.165651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8cfd9-efe4-4cfa-97e0-23eb6b0a6cf4 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:06.165944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8cfd9-efe4-4cfa-97e0-23eb6b0a6cf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:06.167543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b7a8cfd9-efe4-4cfa-97e0-23eb6b0a6cf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:06.264365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8cfd9-efe4-4cfa-97e0-23eb6b0a6cf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:06.266030 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8cfd9-efe4-4cfa-97e0-23eb6b0a6cf4 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:06.285840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 676/1377] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:08.654134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc03a8-cc14-4cdd-bbc8-5e1a2675c78a None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:08.654767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc03a8-cc14-4cdd-bbc8-5e1a2675c78a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:08.654767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc03a8-cc14-4cdd-bbc8-5e1a2675c78a None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:08.655997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 702/1378] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:08.661545 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8c328-216d-4794-a300-9d151cdb6166 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:08.661968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8c328-216d-4794-a300-9d151cdb6166 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:08.662200 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd8c328-216d-4794-a300-9d151cdb6166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:08.752309 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-6cd8c328-216d-4794-a300-9d151cdb6166 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:08.753869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6cd8c328-216d-4794-a300-9d151cdb6166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:08.831944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:08.864946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 677/1379] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:47:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:09.152892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc03a8-cc14-4cdd-bbc8-5e1a2675c78a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:09.155042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7ecc03a8-cc14-4cdd-bbc8-5e1a2675c78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:09.312447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc03a8-cc14-4cdd-bbc8-5e1a2675c78a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:09.326777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:09.429636 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597df49e-f25c-4293-826d-808d25e6e7be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:09.430215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597df49e-f25c-4293-826d-808d25e6e7be service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:09.430501 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597df49e-f25c-4293-826d-808d25e6e7be service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:09.430780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597df49e-f25c-4293-826d-808d25e6e7be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:09.432427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-597df49e-f25c-4293-826d-808d25e6e7be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:09.522717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597df49e-f25c-4293-826d-808d25e6e7be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:09.524847 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597df49e-f25c-4293-826d-808d25e6e7be service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:09.542543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 703/1380] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:11.877414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cbd5e-1821-4d05-8e96-a0b52ed09101 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:11.877919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cbd5e-1821-4d05-8e96-a0b52ed09101 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:11.878022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cbd5e-1821-4d05-8e96-a0b52ed09101 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:11.879303 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 678/1381] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:11.885138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0ccc9-e94b-4a76-b013-7592e0d51204 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:11.885705 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0ccc9-e94b-4a76-b013-7592e0d51204 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:11.885705 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0ccc9-e94b-4a76-b013-7592e0d51204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:11.908032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:11.958989 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6ee0ccc9-e94b-4a76-b013-7592e0d51204 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:11.960195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6ee0ccc9-e94b-4a76-b013-7592e0d51204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:12.076377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 704/1382] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:47:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:12.362995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cbd5e-1821-4d05-8e96-a0b52ed09101 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:12.364554 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-fa2cbd5e-1821-4d05-8e96-a0b52ed09101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:12.518483 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cbd5e-1821-4d05-8e96-a0b52ed09101 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:12.571617 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:12.644744 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee986f2b-6f20-4ff2-ba8c-43c86bd7e7c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:12.645404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee986f2b-6f20-4ff2-ba8c-43c86bd7e7c4 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:12.646139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee986f2b-6f20-4ff2-ba8c-43c86bd7e7c4 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:12.646502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee986f2b-6f20-4ff2-ba8c-43c86bd7e7c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:12.649160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ee986f2b-6f20-4ff2-ba8c-43c86bd7e7c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:12.744952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee986f2b-6f20-4ff2-ba8c-43c86bd7e7c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:12.746633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee986f2b-6f20-4ff2-ba8c-43c86bd7e7c4 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:12.765915 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 679/1383] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:14.913081 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11845398-6ea2-4f58-b01e-364456183ab3 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:14.914463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11845398-6ea2-4f58-b01e-364456183ab3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:14.914463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11845398-6ea2-4f58-b01e-364456183ab3 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:14.915852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 705/1384] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:47:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:14.921232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2700914-3542-4466-b7c7-e1d7a52d7581 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:14.921739 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2700914-3542-4466-b7c7-e1d7a52d7581 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:14.921739 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2700914-3542-4466-b7c7-e1d7a52d7581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:14.992135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-f2700914-3542-4466-b7c7-e1d7a52d7581 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:14.993252 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-f2700914-3542-4466-b7c7-e1d7a52d7581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:15.001785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:15.099671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 680/1385] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:15.391972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11845398-6ea2-4f58-b01e-364456183ab3 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:15.393472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-11845398-6ea2-4f58-b01e-364456183ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:15.485838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:15.541619 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11845398-6ea2-4f58-b01e-364456183ab3 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:15.654311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2c575a-9a75-4457-8184-7c57bdfa120f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:15.654909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c575a-9a75-4457-8184-7c57bdfa120f service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:15.655185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c575a-9a75-4457-8184-7c57bdfa120f service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:15.655451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2c575a-9a75-4457-8184-7c57bdfa120f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:15.657025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2f2c575a-9a75-4457-8184-7c57bdfa120f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:15.718089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:15.750983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c575a-9a75-4457-8184-7c57bdfa120f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:15.752591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2c575a-9a75-4457-8184-7c57bdfa120f service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:15.771323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 706/1386] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:16.306674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de637df-dfb6-4c50-990e-4308b23d0759 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:16.307245 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de637df-dfb6-4c50-990e-4308b23d0759 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:16.307245 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de637df-dfb6-4c50-990e-4308b23d0759 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:16.308574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 681/1387] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:16.314035 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd354775-a27e-4627-ab92-2bb753c4eb6a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:16.314466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd354775-a27e-4627-ab92-2bb753c4eb6a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:16.314574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd354775-a27e-4627-ab92-2bb753c4eb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:16.381190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-fd354775-a27e-4627-ab92-2bb753c4eb6a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:16.382425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fd354775-a27e-4627-ab92-2bb753c4eb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:16.479594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 707/1388] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:47:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:16.777949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe7c69-eb7a-4cb5-ae64-79a5f3cef448 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:16.778239 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe7c69-eb7a-4cb5-ae64-79a5f3cef448 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:16.778514 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe7c69-eb7a-4cb5-ae64-79a5f3cef448 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:16.779842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 682/1389] 10.222.0.42 () {54 vars in 962 bytes} [Wed Sep 14 19:47:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:47:16.785008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb06418-96cb-48ac-b5e4-ee01e6034ffc None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:16.785316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb06418-96cb-48ac-b5e4-ee01e6034ffc None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:16.785587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb06418-96cb-48ac-b5e4-ee01e6034ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:16.832087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:16.866124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-ebb06418-96cb-48ac-b5e4-ee01e6034ffc None None] MFA Rules not processed for user `4e021a00524341359bc4b5a800a9bbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:16.868025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ebb06418-96cb-48ac-b5e4-ee01e6034ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:16.946239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 708/1390] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Sep 14 19:47:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:16.955361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe7c69-eb7a-4cb5-ae64-79a5f3cef448 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:16.956878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8dfe7c69-eb7a-4cb5-ae64-79a5f3cef448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:17.059950 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe7c69-eb7a-4cb5-ae64-79a5f3cef448 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:17.098347 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:17.134308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d34926-d17e-4408-ba66-b40ef1231496 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e021a00524341359bc4b5a800a9bbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:17.134910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d34926-d17e-4408-ba66-b40ef1231496 service glance] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:17.135178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d34926-d17e-4408-ba66-b40ef1231496 service glance] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:17.135450 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d34926-d17e-4408-ba66-b40ef1231496 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:17.137014 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-12d34926-d17e-4408-ba66-b40ef1231496 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:17.245402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d34926-d17e-4408-ba66-b40ef1231496 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:17.246870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d34926-d17e-4408-ba66-b40ef1231496 service glance] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:17.265085 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 683/1391] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:18.139177 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50114922-20ce-470f-bb2c-8ff4c007311c None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:18.139728 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50114922-20ce-470f-bb2c-8ff4c007311c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:18.139728 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50114922-20ce-470f-bb2c-8ff4c007311c None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:18.141025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 709/1392] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:18.146331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d3126-6620-45f8-aebb-447a88ed7482 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:18.146818 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d3126-6620-45f8-aebb-447a88ed7482 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:18.146902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d3126-6620-45f8-aebb-447a88ed7482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:18.177204 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94bb68-975b-45ca-a052-e336d1c21945 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:18.177489 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94bb68-975b-45ca-a052-e336d1c21945 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:18.177766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94bb68-975b-45ca-a052-e336d1c21945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:18.214405 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-345d3126-6620-45f8-aebb-447a88ed7482 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:18.215611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-345d3126-6620-45f8-aebb-447a88ed7482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:18.234721 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:18.245886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-eb94bb68-975b-45ca-a052-e336d1c21945 None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:18.247094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-eb94bb68-975b-45ca-a052-e336d1c21945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:18.314302 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 684/1393] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:47:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:18.327796 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:18.334903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 710/1394] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 19:47:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:18.381392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acb4b1-f4b7-415a-8c94-97f0b914788c None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:18.381681 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acb4b1-f4b7-415a-8c94-97f0b914788c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:18.381998 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22acb4b1-f4b7-415a-8c94-97f0b914788c None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:18.383889 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 685/1395] 10.222.0.42 () {54 vars in 943 bytes} [Wed Sep 14 19:47:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:18.390754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204970f-f64c-476d-9af5-bc126125d284 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:18.391042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204970f-f64c-476d-9af5-bc126125d284 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:18.391316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0204970f-f64c-476d-9af5-bc126125d284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:18.467365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0204970f-f64c-476d-9af5-bc126125d284 None None] MFA Rules not processed for user `e3d2f1ab1bfd435183e33168400c0b17`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:18.468564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0204970f-f64c-476d-9af5-bc126125d284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:18.574447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 711/1396] 10.222.0.42 () {58 vars in 1040 bytes} [Wed Sep 14 19:47:18 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:18.583825 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acb4b1-f4b7-415a-8c94-97f0b914788c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:18.585378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-22acb4b1-f4b7-415a-8c94-97f0b914788c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:18.603680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0204970f-f64c-476d-9af5-bc126125d284 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:18.604871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0204970f-f64c-476d-9af5-bc126125d284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:18.660720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:18.706094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0204970f-f64c-476d-9af5-bc126125d284 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:18.723815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22acb4b1-f4b7-415a-8c94-97f0b914788c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:18.736831 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:18.822120 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6202f-e85d-469c-84fc-401c9f653471 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:18.822725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6202f-e85d-469c-84fc-401c9f653471 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:18.823000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6202f-e85d-469c-84fc-401c9f653471 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:18.823265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6202f-e85d-469c-84fc-401c9f653471 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:18.824908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e8f6202f-e85d-469c-84fc-401c9f653471 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:18.825990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89a2aa-173f-4641-9087-3c4ae288d5ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:18.826690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89a2aa-173f-4641-9087-3c4ae288d5ef service swift] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:18.826966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89a2aa-173f-4641-9087-3c4ae288d5ef service swift] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:18.827227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89a2aa-173f-4641-9087-3c4ae288d5ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:18.828853 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ed89a2aa-173f-4641-9087-3c4ae288d5ef service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:18.899674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:18.920048 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89a2aa-173f-4641-9087-3c4ae288d5ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:18.921566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89a2aa-173f-4641-9087-3c4ae288d5ef service swift] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:18.930510 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 686/1397] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 19:47:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 19:47:18.931514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6202f-e85d-469c-84fc-401c9f653471 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:18.934566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6202f-e85d-469c-84fc-401c9f653471 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:18.955391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 712/1398] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:47:21.526754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34386048-d543-4177-83ec-cc1fd84ec535 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:21.527378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34386048-d543-4177-83ec-cc1fd84ec535 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:21.527378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34386048-d543-4177-83ec-cc1fd84ec535 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:21.528785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 687/1399] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:21.535871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54327c26-475c-4c80-8691-b16e9d99bd61 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:21.536398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54327c26-475c-4c80-8691-b16e9d99bd61 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:21.546303 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54327c26-475c-4c80-8691-b16e9d99bd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:21.634500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-54327c26-475c-4c80-8691-b16e9d99bd61 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:21.635473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-54327c26-475c-4c80-8691-b16e9d99bd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:21.747511 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 713/1400] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:47:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:22.035992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34386048-d543-4177-83ec-cc1fd84ec535 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:22.037695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-34386048-d543-4177-83ec-cc1fd84ec535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:22.102757 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:22.219312 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34386048-d543-4177-83ec-cc1fd84ec535 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:22.346985 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f3b94-da91-4bc9-962e-ad0c0f86656f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:22.347594 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f3b94-da91-4bc9-962e-ad0c0f86656f service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:22.347868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f3b94-da91-4bc9-962e-ad0c0f86656f service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:22.348283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f3b94-da91-4bc9-962e-ad0c0f86656f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:22.350018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-561f3b94-da91-4bc9-962e-ad0c0f86656f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:22.357343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:22.458304 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f3b94-da91-4bc9-962e-ad0c0f86656f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:22.460801 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f3b94-da91-4bc9-962e-ad0c0f86656f service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:22.481348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 688/1401] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:24.858886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456be23b-cd2d-4142-a0f8-fdb3641915d0 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:24.859930 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456be23b-cd2d-4142-a0f8-fdb3641915d0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:24.860229 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456be23b-cd2d-4142-a0f8-fdb3641915d0 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:24.861561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 714/1402] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:47:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:24.868045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df573d01-7867-4101-94da-5de630bfcbdb None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:24.868563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df573d01-7867-4101-94da-5de630bfcbdb None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:24.868563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df573d01-7867-4101-94da-5de630bfcbdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:24.944069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-df573d01-7867-4101-94da-5de630bfcbdb None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:24.945490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-df573d01-7867-4101-94da-5de630bfcbdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:25.029113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:25.055645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 689/1403] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:25.362206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456be23b-cd2d-4142-a0f8-fdb3641915d0 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:25.363678 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-456be23b-cd2d-4142-a0f8-fdb3641915d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:25.412835 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:25.529897 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456be23b-cd2d-4142-a0f8-fdb3641915d0 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:25.653281 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfc818d-4b4d-4015-9a1b-881d7e1665c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:25.653955 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc818d-4b4d-4015-9a1b-881d7e1665c3 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:25.654232 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc818d-4b4d-4015-9a1b-881d7e1665c3 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:25.654501 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfc818d-4b4d-4015-9a1b-881d7e1665c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:25.656102 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3bfc818d-4b4d-4015-9a1b-881d7e1665c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:25.658398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:25.758652 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc818d-4b4d-4015-9a1b-881d7e1665c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:25.760305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfc818d-4b4d-4015-9a1b-881d7e1665c3 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:25.778751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 715/1404] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:28.219190 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadb688-f1e2-4ef8-9cfd-638921f4949e None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:28.219780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadb688-f1e2-4ef8-9cfd-638921f4949e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:28.219780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadb688-f1e2-4ef8-9cfd-638921f4949e None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:28.226734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 690/1405] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:47:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:28.227376 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d143f-ba79-42b4-a7ac-1b4dadd89e87 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:28.227818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d143f-ba79-42b4-a7ac-1b4dadd89e87 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:28.227971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d143f-ba79-42b4-a7ac-1b4dadd89e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:28.302147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-df5d143f-ba79-42b4-a7ac-1b4dadd89e87 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:28.303392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-df5d143f-ba79-42b4-a7ac-1b4dadd89e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:28.387831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:28.414607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 716/1406] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:28.703445 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadb688-f1e2-4ef8-9cfd-638921f4949e None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:28.704969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-daadb688-f1e2-4ef8-9cfd-638921f4949e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:28.872259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadb688-f1e2-4ef8-9cfd-638921f4949e None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:28.940530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:28.998659 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b49e5d3-fe2f-4d34-a876-f0d7f0ebafd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:28.999268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49e5d3-fe2f-4d34-a876-f0d7f0ebafd0 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:28.999544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49e5d3-fe2f-4d34-a876-f0d7f0ebafd0 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:28.999814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b49e5d3-fe2f-4d34-a876-f0d7f0ebafd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:29.001415 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4b49e5d3-fe2f-4d34-a876-f0d7f0ebafd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:29.103310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49e5d3-fe2f-4d34-a876-f0d7f0ebafd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:29.105630 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b49e5d3-fe2f-4d34-a876-f0d7f0ebafd0 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:29.124123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 691/1407] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:38.418034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=9 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:39.121788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=69 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:43.243008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7819bc-2384-4e37-8444-9791ef898e90 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:43.243575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7819bc-2384-4e37-8444-9791ef898e90 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:43.243575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7819bc-2384-4e37-8444-9791ef898e90 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:43.244827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 717/1408] 10.222.0.42 () {54 vars in 918 bytes} [Wed Sep 14 19:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:43.249904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11f971-fc38-4fdb-b090-0c4126d4821a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:43.250353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11f971-fc38-4fdb-b090-0c4126d4821a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:43.250453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11f971-fc38-4fdb-b090-0c4126d4821a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:43.324383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-0b11f971-fc38-4fdb-b090-0c4126d4821a None None] MFA Rules not processed for user `03bc280d0395479981b7c7b08c96001b`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:43.325598 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0b11f971-fc38-4fdb-b090-0c4126d4821a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:43.429119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 692/1409] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Sep 14 19:47:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:43.443548 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66c755-a67c-4609-b2e8-db3b9e46bd43 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:43.443826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66c755-a67c-4609-b2e8-db3b9e46bd43 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:43.444095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d66c755-a67c-4609-b2e8-db3b9e46bd43 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:43.445449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 718/1410] 10.222.0.42 () {54 vars in 918 bytes} [Wed Sep 14 19:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:43.450077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6975d-6167-460e-a1a0-60b0e75153c9 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:43.450358 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6975d-6167-460e-a1a0-60b0e75153c9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:43.450629 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f6975d-6167-460e-a1a0-60b0e75153c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:43.522290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:43.525178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-86f6975d-6167-460e-a1a0-60b0e75153c9 None None] MFA Rules not processed for user `2a4284b59f804dc59674d621d87634bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:43.526396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-86f6975d-6167-460e-a1a0-60b0e75153c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:43.641198 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 693/1411] 10.222.0.42 () {58 vars in 1015 bytes} [Wed Sep 14 19:47:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:46.183374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe270d-0be8-45f9-983d-cb2d8fa5b718 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:46.183937 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe270d-0be8-45f9-983d-cb2d8fa5b718 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:46.183937 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe270d-0be8-45f9-983d-cb2d8fa5b718 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:46.185235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 719/1412] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:46.190494 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a81f0-0a79-4724-9d1a-d0aebfa7459c None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:46.190957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a81f0-0a79-4724-9d1a-d0aebfa7459c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:46.191346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441a81f0-0a79-4724-9d1a-d0aebfa7459c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:46.256744 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-441a81f0-0a79-4724-9d1a-d0aebfa7459c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:46.257966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-441a81f0-0a79-4724-9d1a-d0aebfa7459c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:46.310826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:46.351449 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 694/1413] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:46.638552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe270d-0be8-45f9-983d-cb2d8fa5b718 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:46.640075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1efe270d-0be8-45f9-983d-cb2d8fa5b718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:46.783548 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe270d-0be8-45f9-983d-cb2d8fa5b718 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:46.855387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:46.897148 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c8fa7-cfd7-48b7-b015-5e8978004f87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:46.897687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c8fa7-cfd7-48b7-b015-5e8978004f87 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:46.897991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c8fa7-cfd7-48b7-b015-5e8978004f87 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:46.898260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c8fa7-cfd7-48b7-b015-5e8978004f87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:46.899865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-399c8fa7-cfd7-48b7-b015-5e8978004f87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:46.986716 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c8fa7-cfd7-48b7-b015-5e8978004f87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:46.988328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c8fa7-cfd7-48b7-b015-5e8978004f87 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:47.006348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 720/1414] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:48.149714 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52a8db-0e34-49a3-99bf-cb78e6d410c7 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:48.150316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52a8db-0e34-49a3-99bf-cb78e6d410c7 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:48.150316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52a8db-0e34-49a3-99bf-cb78e6d410c7 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:48.151651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 695/1415] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 19:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:48.157923 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39ca5b-ee38-41b6-bb81-f771e87da199 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:48.158354 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39ca5b-ee38-41b6-bb81-f771e87da199 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:48.158478 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39ca5b-ee38-41b6-bb81-f771e87da199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:48.233140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2c39ca5b-ee38-41b6-bb81-f771e87da199 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:48.234368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2c39ca5b-ee38-41b6-bb81-f771e87da199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:48.260955 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:48.329557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 721/1416] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 19:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:48.479637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52a8db-0e34-49a3-99bf-cb78e6d410c7 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:48.481109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-fc52a8db-0e34-49a3-99bf-cb78e6d410c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:48.624396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52a8db-0e34-49a3-99bf-cb78e6d410c7 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:48.664074 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:48.735074 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d9190-85e6-442e-8ccb-97a060f91ea8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:48.735623 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d9190-85e6-442e-8ccb-97a060f91ea8 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:48.735892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d9190-85e6-442e-8ccb-97a060f91ea8 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:48.736157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d9190-85e6-442e-8ccb-97a060f91ea8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:48.737721 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b21d9190-85e6-442e-8ccb-97a060f91ea8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:48.825479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d9190-85e6-442e-8ccb-97a060f91ea8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:48.827105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d9190-85e6-442e-8ccb-97a060f91ea8 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:48.844436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 696/1417] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:49.973127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e1645-fd4a-417a-a373-408abacb2f79 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:49.973694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e1645-fd4a-417a-a373-408abacb2f79 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:49.973694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e1645-fd4a-417a-a373-408abacb2f79 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:49.975578 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 722/1418] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 19:47:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:49.981823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766754d8-0c58-4133-8c94-cbc7d23f5309 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:49.982264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766754d8-0c58-4133-8c94-cbc7d23f5309 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:49.982438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766754d8-0c58-4133-8c94-cbc7d23f5309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:50.036124 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:50.057381 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-766754d8-0c58-4133-8c94-cbc7d23f5309 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:50.058608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-766754d8-0c58-4133-8c94-cbc7d23f5309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:50.152045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 697/1419] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 19:47:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:50.301620 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e1645-fd4a-417a-a373-408abacb2f79 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:50.303673 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7f8e1645-fd4a-417a-a373-408abacb2f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:50.447227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e1645-fd4a-417a-a373-408abacb2f79 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:50.459117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:50.560229 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d0f2b-e154-4e3f-9470-76b729a2d5a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:50.560749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d0f2b-e154-4e3f-9470-76b729a2d5a1 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:50.561022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d0f2b-e154-4e3f-9470-76b729a2d5a1 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:50.561289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d0f2b-e154-4e3f-9470-76b729a2d5a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:50.562911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-065d0f2b-e154-4e3f-9470-76b729a2d5a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:50.649746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d0f2b-e154-4e3f-9470-76b729a2d5a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:50.651356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d0f2b-e154-4e3f-9470-76b729a2d5a1 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:50.669134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 723/1420] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:51.770320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02e515-d58b-4704-ba1a-9a3bda9dbef7 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:51.770775 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02e515-d58b-4704-ba1a-9a3bda9dbef7 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:51.770871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02e515-d58b-4704-ba1a-9a3bda9dbef7 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:51.772279 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 698/1421] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 19:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:51.778476 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabba3b2-ef93-4bb5-bbb5-1272ddf61fce None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:51.778871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabba3b2-ef93-4bb5-bbb5-1272ddf61fce None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:51.779002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabba3b2-ef93-4bb5-bbb5-1272ddf61fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:51.798136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:51.854421 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-fabba3b2-ef93-4bb5-bbb5-1272ddf61fce None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:51.855632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fabba3b2-ef93-4bb5-bbb5-1272ddf61fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:51.951169 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 724/1422] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 19:47:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:52.100383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02e515-d58b-4704-ba1a-9a3bda9dbef7 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:52.101966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1a02e515-d58b-4704-ba1a-9a3bda9dbef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:52.210336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:52.246021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02e515-d58b-4704-ba1a-9a3bda9dbef7 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:52.355992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ed9551-54c5-48f6-8b93-17cdd0248247 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:52.356520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed9551-54c5-48f6-8b93-17cdd0248247 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:52.356796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed9551-54c5-48f6-8b93-17cdd0248247 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:52.357065 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ed9551-54c5-48f6-8b93-17cdd0248247 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:52.358673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c5ed9551-54c5-48f6-8b93-17cdd0248247 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:52.434185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:52.447975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed9551-54c5-48f6-8b93-17cdd0248247 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:52.449555 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ed9551-54c5-48f6-8b93-17cdd0248247 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:52.467910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 699/1423] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:53.571717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aad1c-5e2e-429c-b1f5-82ae4746cffc None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:53.572231 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aad1c-5e2e-429c-b1f5-82ae4746cffc None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:53.572337 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6aad1c-5e2e-429c-b1f5-82ae4746cffc None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:53.573591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 725/1424] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 19:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:53.580055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaab0ef-c909-4a61-855c-94b33f150da7 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:53.580615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaab0ef-c909-4a61-855c-94b33f150da7 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:53.580615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaab0ef-c909-4a61-855c-94b33f150da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:53.656715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-daaab0ef-c909-4a61-855c-94b33f150da7 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:53.658018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-daaab0ef-c909-4a61-855c-94b33f150da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:53.755560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 700/1425] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 19:47:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:53.904963 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6aad1c-5e2e-429c-b1f5-82ae4746cffc None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:53.906513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ff6aad1c-5e2e-429c-b1f5-82ae4746cffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:53.989255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:54.049649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6aad1c-5e2e-429c-b1f5-82ae4746cffc None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:54.163140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7d200-1df1-4b8e-a8c4-76553fd76d3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:54.163663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7d200-1df1-4b8e-a8c4-76553fd76d3f service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:54.163937 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7d200-1df1-4b8e-a8c4-76553fd76d3f service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:54.164203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7d200-1df1-4b8e-a8c4-76553fd76d3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:54.165825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-70c7d200-1df1-4b8e-a8c4-76553fd76d3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:54.216745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:54.256300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7d200-1df1-4b8e-a8c4-76553fd76d3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:54.257897 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7d200-1df1-4b8e-a8c4-76553fd76d3f service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:54.275635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 726/1426] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:55.386728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3424c-8d5d-4c09-ab95-01486a1403a9 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:55.387294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3424c-8d5d-4c09-ab95-01486a1403a9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:55.387294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3424c-8d5d-4c09-ab95-01486a1403a9 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:55.388675 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 701/1427] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 19:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:55.394844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eebe26-275f-43c3-b9e9-d5855b180842 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:55.395344 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eebe26-275f-43c3-b9e9-d5855b180842 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:55.395440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eebe26-275f-43c3-b9e9-d5855b180842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:55.469336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f8eebe26-275f-43c3-b9e9-d5855b180842 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:55.470564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f8eebe26-275f-43c3-b9e9-d5855b180842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:55.571146 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 727/1428] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 19:47:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:55.720401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3424c-8d5d-4c09-ab95-01486a1403a9 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:55.721964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e0e3424c-8d5d-4c09-ab95-01486a1403a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:55.771003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:55.866276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3424c-8d5d-4c09-ab95-01486a1403a9 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:55.977043 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cee743b-3e1c-4df4-a7a9-39135c2ba498 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:55.977636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee743b-3e1c-4df4-a7a9-39135c2ba498 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:55.977939 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee743b-3e1c-4df4-a7a9-39135c2ba498 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:55.978204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cee743b-3e1c-4df4-a7a9-39135c2ba498 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:55.979767 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2cee743b-3e1c-4df4-a7a9-39135c2ba498 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:55.986473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:56.068778 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee743b-3e1c-4df4-a7a9-39135c2ba498 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:56.070383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cee743b-3e1c-4df4-a7a9-39135c2ba498 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:56.088518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 702/1429] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:57.195379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ddcc2-8c25-41e8-9bf0-1fd1ccca3e02 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:57.195911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ddcc2-8c25-41e8-9bf0-1fd1ccca3e02 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:57.195911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24ddcc2-8c25-41e8-9bf0-1fd1ccca3e02 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:57.197160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 728/1430] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 19:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:57.203426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f98e2-1003-47a7-aa9d-c996a5992317 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:57.203842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f98e2-1003-47a7-aa9d-c996a5992317 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:57.203948 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f98e2-1003-47a7-aa9d-c996a5992317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:57.277409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-846f98e2-1003-47a7-aa9d-c996a5992317 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:57.278644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-846f98e2-1003-47a7-aa9d-c996a5992317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:57.344249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:57.372823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 703/1431] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 19:47:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:57.522251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ddcc2-8c25-41e8-9bf0-1fd1ccca3e02 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:57.523675 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a24ddcc2-8c25-41e8-9bf0-1fd1ccca3e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:57.541681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:57.667577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24ddcc2-8c25-41e8-9bf0-1fd1ccca3e02 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:57.753657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:57.780267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de471d1-9357-4266-b670-d59eba0c020b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:57.780812 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de471d1-9357-4266-b670-d59eba0c020b service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:57.781085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de471d1-9357-4266-b670-d59eba0c020b service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:57.781405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de471d1-9357-4266-b670-d59eba0c020b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:57.783126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0de471d1-9357-4266-b670-d59eba0c020b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:57.871886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de471d1-9357-4266-b670-d59eba0c020b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:57.873517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de471d1-9357-4266-b670-d59eba0c020b service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:57.891751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 729/1432] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:47:58.994516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81220e-d41d-444a-8710-bf7e0afa2ee5 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:58.995058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81220e-d41d-444a-8710-bf7e0afa2ee5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:58.995058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f81220e-d41d-444a-8710-bf7e0afa2ee5 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:58.996406 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 704/1433] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 19:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:47:59.002639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317d713-6ff8-4e92-9e91-edff40834860 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:59.003075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317d713-6ff8-4e92-9e91-edff40834860 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:59.003194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4317d713-6ff8-4e92-9e91-edff40834860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:59.079085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4317d713-6ff8-4e92-9e91-edff40834860 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:47:59.080287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4317d713-6ff8-4e92-9e91-edff40834860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:59.120810 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:59.173642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 730/1434] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 19:47:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:47:59.324059 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81220e-d41d-444a-8710-bf7e0afa2ee5 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:47:59.325552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1f81220e-d41d-444a-8710-bf7e0afa2ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:59.466363 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f81220e-d41d-444a-8710-bf7e0afa2ee5 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:47:59.518784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:47:59.576313 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550eee9-5edc-48c3-964e-c407c98b0131 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:47:59.576907 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550eee9-5edc-48c3-964e-c407c98b0131 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:47:59.577177 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550eee9-5edc-48c3-964e-c407c98b0131 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:47:59.577441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550eee9-5edc-48c3-964e-c407c98b0131 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:47:59.579104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-7550eee9-5edc-48c3-964e-c407c98b0131 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:47:59.666946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550eee9-5edc-48c3-964e-c407c98b0131 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:47:59.668515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550eee9-5edc-48c3-964e-c407c98b0131 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:47:59.686243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 705/1435] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:47:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:48:00.797983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a4c9a-1f25-45a1-8035-b4d9a97babb5 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:00.798568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a4c9a-1f25-45a1-8035-b4d9a97babb5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:00.798568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a4c9a-1f25-45a1-8035-b4d9a97babb5 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:00.799909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 731/1436] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 19:48:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:00.806556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075c0db-d477-45f6-b35f-fd9f53097d8e None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:00.806960 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075c0db-d477-45f6-b35f-fd9f53097d8e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:00.807102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1075c0db-d477-45f6-b35f-fd9f53097d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:00.883811 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-1075c0db-d477-45f6-b35f-fd9f53097d8e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:00.885023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1075c0db-d477-45f6-b35f-fd9f53097d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:00.887053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:00.978880 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 706/1437] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 19:48:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:01.129471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a4c9a-1f25-45a1-8035-b4d9a97babb5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:48:01.130963 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1f8a4c9a-1f25-45a1-8035-b4d9a97babb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:01.273827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a4c9a-1f25-45a1-8035-b4d9a97babb5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:48:01.301998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:01.385152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e883e73-9f1a-42f6-9441-2f406cb5ca84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:48:01.385678 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e883e73-9f1a-42f6-9441-2f406cb5ca84 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:01.385988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e883e73-9f1a-42f6-9441-2f406cb5ca84 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:01.386253 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e883e73-9f1a-42f6-9441-2f406cb5ca84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:01.387846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1e883e73-9f1a-42f6-9441-2f406cb5ca84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:01.475142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e883e73-9f1a-42f6-9441-2f406cb5ca84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:48:01.476719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e883e73-9f1a-42f6-9441-2f406cb5ca84 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:48:01.494079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 732/1438] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:48:03.526742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a8b3c-4ef9-4703-83b7-2dd9923000fd None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:03.527338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a8b3c-4ef9-4703-83b7-2dd9923000fd None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:03.527338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a8b3c-4ef9-4703-83b7-2dd9923000fd None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:03.528651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 707/1439] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:03.534034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e088a4-198f-44e0-87fe-2b75889896d3 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:03.534444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e088a4-198f-44e0-87fe-2b75889896d3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:03.534582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e088a4-198f-44e0-87fe-2b75889896d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:03.576765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:03.601612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d3e088a4-198f-44e0-87fe-2b75889896d3 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:03.602855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d3e088a4-198f-44e0-87fe-2b75889896d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:03.697137 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 733/1440] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:48:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:03.981518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a8b3c-4ef9-4703-83b7-2dd9923000fd None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:48:03.983112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-802a8b3c-4ef9-4703-83b7-2dd9923000fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:04.084985 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a8b3c-4ef9-4703-83b7-2dd9923000fd None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:48:04.116320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:04.156626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafc7354-be41-4f73-bb8e-97d08def6b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:48:04.158007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc7354-be41-4f73-bb8e-97d08def6b07 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:04.158575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc7354-be41-4f73-bb8e-97d08def6b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:04.159031 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafc7354-be41-4f73-bb8e-97d08def6b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:04.160989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bafc7354-be41-4f73-bb8e-97d08def6b07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:04.268645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc7354-be41-4f73-bb8e-97d08def6b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:48:04.270104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafc7354-be41-4f73-bb8e-97d08def6b07 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:48:04.288306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 708/1441] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:48:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:48:06.546712 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7fd2f-514d-4f7c-96e2-c5f667e92ef9 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:06.547261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7fd2f-514d-4f7c-96e2-c5f667e92ef9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:06.547261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7fd2f-514d-4f7c-96e2-c5f667e92ef9 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:06.548595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 734/1442] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:06.554538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05c361-7e0a-435a-99a8-484323169a30 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:06.554989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05c361-7e0a-435a-99a8-484323169a30 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:06.555106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05c361-7e0a-435a-99a8-484323169a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:06.619720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-af05c361-7e0a-435a-99a8-484323169a30 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:06.620918 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-af05c361-7e0a-435a-99a8-484323169a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:06.634080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:06.715851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 709/1443] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:48:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:06.997238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7fd2f-514d-4f7c-96e2-c5f667e92ef9 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:48:06.998794 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4fe7fd2f-514d-4f7c-96e2-c5f667e92ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:07.101209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7fd2f-514d-4f7c-96e2-c5f667e92ef9 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:48:07.113828 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:07.174405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f74d81-8fcd-49d2-a361-88df83ae0118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:48:07.174938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f74d81-8fcd-49d2-a361-88df83ae0118 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:07.175214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f74d81-8fcd-49d2-a361-88df83ae0118 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:07.175483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f74d81-8fcd-49d2-a361-88df83ae0118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:07.177086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-53f74d81-8fcd-49d2-a361-88df83ae0118 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:07.286866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f74d81-8fcd-49d2-a361-88df83ae0118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:48:07.288274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f74d81-8fcd-49d2-a361-88df83ae0118 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:48:07.306876 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 735/1444] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:48:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:48:09.789718 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2a2c7-d5ef-47f7-92ff-0abb7b69c749 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:09.790188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2a2c7-d5ef-47f7-92ff-0abb7b69c749 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:09.790301 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf2a2c7-d5ef-47f7-92ff-0abb7b69c749 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:09.791668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 710/1445] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:09.796990 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007a646-f1d7-4398-9062-fd102f93d4bb None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:09.797439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007a646-f1d7-4398-9062-fd102f93d4bb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:09.797538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3007a646-f1d7-4398-9062-fd102f93d4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:09.845630 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:09.863423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3007a646-f1d7-4398-9062-fd102f93d4bb None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:09.865140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3007a646-f1d7-4398-9062-fd102f93d4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:09.960660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 736/1446] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:10.244661 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2a2c7-d5ef-47f7-92ff-0abb7b69c749 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:48:10.246275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bdf2a2c7-d5ef-47f7-92ff-0abb7b69c749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:10.348058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf2a2c7-d5ef-47f7-92ff-0abb7b69c749 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:48:10.387520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:10.421011 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3b498a-137b-4871-9c31-5e5cbd81727c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:48:10.421617 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b498a-137b-4871-9c31-5e5cbd81727c service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:10.421936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b498a-137b-4871-9c31-5e5cbd81727c service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:10.422210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3b498a-137b-4871-9c31-5e5cbd81727c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:10.423838 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-3c3b498a-137b-4871-9c31-5e5cbd81727c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:10.530779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b498a-137b-4871-9c31-5e5cbd81727c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:48:10.532144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3b498a-137b-4871-9c31-5e5cbd81727c service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:48:10.551964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 711/1447] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:48:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:48:12.824572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f937f8-9211-4f89-8f73-d55434085f1b None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:12.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f937f8-9211-4f89-8f73-d55434085f1b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:12.825167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f937f8-9211-4f89-8f73-d55434085f1b None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:12.826538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 737/1448] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:12.832036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710517a-76a2-48ab-928a-b966110a439e None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:12.832471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710517a-76a2-48ab-928a-b966110a439e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:12.832586 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0710517a-76a2-48ab-928a-b966110a439e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:12.897974 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-0710517a-76a2-48ab-928a-b966110a439e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:12.899204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0710517a-76a2-48ab-928a-b966110a439e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:12.919723 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:12.994069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 712/1449] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:13.275809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f937f8-9211-4f89-8f73-d55434085f1b None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:48:13.277353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-64f937f8-9211-4f89-8f73-d55434085f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:13.378888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f937f8-9211-4f89-8f73-d55434085f1b None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:48:13.399411 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:13.452586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb503e8-8b03-47fe-a2e4-e5bb662862f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:48:13.453188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb503e8-8b03-47fe-a2e4-e5bb662862f1 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:13.453466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb503e8-8b03-47fe-a2e4-e5bb662862f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:13.453736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb503e8-8b03-47fe-a2e4-e5bb662862f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:13.455341 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-deb503e8-8b03-47fe-a2e4-e5bb662862f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:13.680539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb503e8-8b03-47fe-a2e4-e5bb662862f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:48:13.681980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb503e8-8b03-47fe-a2e4-e5bb662862f1 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:48:13.699961 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 738/1450] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:48:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:48:16.033892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a5240-665b-4c42-8b27-89fe7ac09d67 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:16.034441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a5240-665b-4c42-8b27-89fe7ac09d67 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:16.034441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a5240-665b-4c42-8b27-89fe7ac09d67 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:16.035694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 713/1451] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:16.041086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022300bc-37a8-4524-9a98-4570b75ee7cb None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:16.041537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022300bc-37a8-4524-9a98-4570b75ee7cb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:16.041633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022300bc-37a8-4524-9a98-4570b75ee7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:16.103846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:16.106350 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-022300bc-37a8-4524-9a98-4570b75ee7cb None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:16.107555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-022300bc-37a8-4524-9a98-4570b75ee7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:16.201291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 739/1452] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:48:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:16.485208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a5240-665b-4c42-8b27-89fe7ac09d67 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:48:16.486791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-775a5240-665b-4c42-8b27-89fe7ac09d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:16.593267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a5240-665b-4c42-8b27-89fe7ac09d67 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:48:16.641968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:16.665564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359bc6ba-062d-4547-bf8a-1dd0685ed74f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:48:16.666247 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bc6ba-062d-4547-bf8a-1dd0685ed74f service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:16.666549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bc6ba-062d-4547-bf8a-1dd0685ed74f service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:16.666823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359bc6ba-062d-4547-bf8a-1dd0685ed74f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:16.668409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-359bc6ba-062d-4547-bf8a-1dd0685ed74f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:16.773571 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bc6ba-062d-4547-bf8a-1dd0685ed74f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:48:16.774953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359bc6ba-062d-4547-bf8a-1dd0685ed74f service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:48:16.792699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 714/1453] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:48:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:48:18.961394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be9b73-d828-40d8-8430-4dabaed8bfb5 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:18.961953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be9b73-d828-40d8-8430-4dabaed8bfb5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:18.962050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77be9b73-d828-40d8-8430-4dabaed8bfb5 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:18.963361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 740/1454] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:18.968938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751948a9-eaa8-4a39-9c3e-93cfbb3e26cc None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:18.969358 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751948a9-eaa8-4a39-9c3e-93cfbb3e26cc None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:18.969484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751948a9-eaa8-4a39-9c3e-93cfbb3e26cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:19.037223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-751948a9-eaa8-4a39-9c3e-93cfbb3e26cc None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:19.038440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-751948a9-eaa8-4a39-9c3e-93cfbb3e26cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:19.066439 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:19.134566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 715/1455] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:19.421419 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be9b73-d828-40d8-8430-4dabaed8bfb5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:48:19.423069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-77be9b73-d828-40d8-8430-4dabaed8bfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:19.524494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77be9b73-d828-40d8-8430-4dabaed8bfb5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:48:19.553213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:19.596879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86872b17-e3c4-44fc-b379-fc23f29812df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:48:19.597477 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86872b17-e3c4-44fc-b379-fc23f29812df service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:19.597747 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86872b17-e3c4-44fc-b379-fc23f29812df service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:19.598045 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86872b17-e3c4-44fc-b379-fc23f29812df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:19.599607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-86872b17-e3c4-44fc-b379-fc23f29812df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:19.707082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86872b17-e3c4-44fc-b379-fc23f29812df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:48:19.708445 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86872b17-e3c4-44fc-b379-fc23f29812df service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:48:19.726683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 741/1456] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:48:21.918493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19459d-0b2c-46c7-890e-3b07a1981544 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:21.918969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19459d-0b2c-46c7-890e-3b07a1981544 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:21.919086 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19459d-0b2c-46c7-890e-3b07a1981544 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:21.920364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 716/1457] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:21.925751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318b073-2da5-4f52-9626-a50b25dec67e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:21.926216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318b073-2da5-4f52-9626-a50b25dec67e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:21.926328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318b073-2da5-4f52-9626-a50b25dec67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:21.990552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c318b073-2da5-4f52-9626-a50b25dec67e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:21.991743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c318b073-2da5-4f52-9626-a50b25dec67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:21.999770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:22.086351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 742/1458] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:22.368062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19459d-0b2c-46c7-890e-3b07a1981544 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:48:22.369637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8f19459d-0b2c-46c7-890e-3b07a1981544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:22.471841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19459d-0b2c-46c7-890e-3b07a1981544 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:48:22.525852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:22.543661 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec64acc-cda3-4332-8752-361d123c82c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:48:22.544259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec64acc-cda3-4332-8752-361d123c82c6 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:22.544650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec64acc-cda3-4332-8752-361d123c82c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:22.544918 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec64acc-cda3-4332-8752-361d123c82c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:22.546531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6ec64acc-cda3-4332-8752-361d123c82c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:22.652855 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec64acc-cda3-4332-8752-361d123c82c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:48:22.654247 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec64acc-cda3-4332-8752-361d123c82c6 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:48:22.672560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 717/1459] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:48:24.888803 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d7895-35eb-49ff-b78a-545011a81196 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:24.889288 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d7895-35eb-49ff-b78a-545011a81196 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:24.889396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d7895-35eb-49ff-b78a-545011a81196 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:24.890750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 743/1460] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:24.896701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234d589-0d07-4442-b182-3555509e051b None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:24.897120 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234d589-0d07-4442-b182-3555509e051b None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:24.897226 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234d589-0d07-4442-b182-3555509e051b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:24.962200 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-a234d589-0d07-4442-b182-3555509e051b None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:24.963387 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a234d589-0d07-4442-b182-3555509e051b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:25.000208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:25.056166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 718/1461] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:25.338737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d7895-35eb-49ff-b78a-545011a81196 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:48:25.340311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-245d7895-35eb-49ff-b78a-545011a81196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:25.442596 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d7895-35eb-49ff-b78a-545011a81196 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:48:25.479671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:25.514040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbf702-c515-4a9b-a468-f1d7830a9981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:48:25.514636 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbf702-c515-4a9b-a468-f1d7830a9981 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:25.514907 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbf702-c515-4a9b-a468-f1d7830a9981 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:25.515170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbf702-c515-4a9b-a468-f1d7830a9981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:25.516738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-dafbf702-c515-4a9b-a468-f1d7830a9981 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:25.623174 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafbf702-c515-4a9b-a468-f1d7830a9981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:48:25.624521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafbf702-c515-4a9b-a468-f1d7830a9981 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:48:25.643871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 744/1462] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:48:27.824770 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff66f1-19e5-4dc9-9206-1148169440ac None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:27.825312 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff66f1-19e5-4dc9-9206-1148169440ac None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:27.825312 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff66f1-19e5-4dc9-9206-1148169440ac None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:27.826716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 719/1463] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:27.832052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e75bbe-e2d1-486b-a7ba-4c1eb26525ba None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:27.832535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e75bbe-e2d1-486b-a7ba-4c1eb26525ba None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:27.832638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e75bbe-e2d1-486b-a7ba-4c1eb26525ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:27.897465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-79e75bbe-e2d1-486b-a7ba-4c1eb26525ba None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:27.898759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-79e75bbe-e2d1-486b-a7ba-4c1eb26525ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:27.916500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:27.993422 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 745/1464] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:28.293863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c507e3a-3938-4292-8f8b-ab1b1c728a9a None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:28.294147 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c507e3a-3938-4292-8f8b-ab1b1c728a9a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:28.294416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c507e3a-3938-4292-8f8b-ab1b1c728a9a None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:28.295771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 720/1465] 10.222.0.42 () {54 vars in 965 bytes} [Wed Sep 14 19:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:28.301635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63992fa7-2e17-4327-bbdd-5a8681b6ca34 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:28.301957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63992fa7-2e17-4327-bbdd-5a8681b6ca34 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:28.302230 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63992fa7-2e17-4327-bbdd-5a8681b6ca34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:28.375790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-63992fa7-2e17-4327-bbdd-5a8681b6ca34 None None] MFA Rules not processed for user `7103ff9544524393a8c7175998dcae3b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:28.376996 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-63992fa7-2e17-4327-bbdd-5a8681b6ca34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:28.450325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 746/1466] 10.222.0.42 () {58 vars in 1062 bytes} [Wed Sep 14 19:48:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:28.459993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c507e3a-3938-4292-8f8b-ab1b1c728a9a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:48:28.461564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8c507e3a-3938-4292-8f8b-ab1b1c728a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:28.562489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c507e3a-3938-4292-8f8b-ab1b1c728a9a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:48:28.627642 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:28.634300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5aeec-96f6-4629-be73-050d87847ddb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:48:28.634894 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5aeec-96f6-4629-be73-050d87847ddb service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:28.635174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5aeec-96f6-4629-be73-050d87847ddb service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:28.635445 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5aeec-96f6-4629-be73-050d87847ddb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:28.637015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c0c5aeec-96f6-4629-be73-050d87847ddb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:28.743975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5aeec-96f6-4629-be73-050d87847ddb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:48:28.745400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5aeec-96f6-4629-be73-050d87847ddb service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:48:28.763260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 721/1467] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:48:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:48:31.324648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d561f-d10f-427f-92c1-3af96869f5bb None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:31.325205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d561f-d10f-427f-92c1-3af96869f5bb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:31.325316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d561f-d10f-427f-92c1-3af96869f5bb None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:31.326627 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 747/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:31.332634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca1c0e-6ee9-46b4-8de4-3940ad4bf32b None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:31.333065 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca1c0e-6ee9-46b4-8de4-3940ad4bf32b None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:31.333199 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca1c0e-6ee9-46b4-8de4-3940ad4bf32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:31.398796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-0fca1c0e-6ee9-46b4-8de4-3940ad4bf32b None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:31.399992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0fca1c0e-6ee9-46b4-8de4-3940ad4bf32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:31.447441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:31.494886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 722/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:31.776456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d561f-d10f-427f-92c1-3af96869f5bb None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:48:31.778043 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5f4d561f-d10f-427f-92c1-3af96869f5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:31.792288 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:31.880560 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d561f-d10f-427f-92c1-3af96869f5bb None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:48:31.952895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fb0e6c-736c-4ab8-9a4e-99085d602ced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:48:31.953490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb0e6c-736c-4ab8-9a4e-99085d602ced service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:31.953759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb0e6c-736c-4ab8-9a4e-99085d602ced service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:31.954059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fb0e6c-736c-4ab8-9a4e-99085d602ced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:31.955620 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f3fb0e6c-736c-4ab8-9a4e-99085d602ced service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:32.023596 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:32.063079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb0e6c-736c-4ab8-9a4e-99085d602ced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:48:32.064482 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fb0e6c-736c-4ab8-9a4e-99085d602ced service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:48:32.082588 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 748/1470] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:48:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:48:41.496560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=22 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:42.079807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=25 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:56.187127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95890c08-7771-455d-83e7-a9abae4926cf None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:56.187746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95890c08-7771-455d-83e7-a9abae4926cf None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:56.187746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95890c08-7771-455d-83e7-a9abae4926cf None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:56.189011 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 723/1471] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:56.194873 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f723c-9764-4bd3-b21e-47af6190858c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:56.195337 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f723c-9764-4bd3-b21e-47af6190858c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:56.195442 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f723c-9764-4bd3-b21e-47af6190858c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:56.259505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d39f723c-9764-4bd3-b21e-47af6190858c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:56.260691 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d39f723c-9764-4bd3-b21e-47af6190858c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:56.353243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 749/1472] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:56.660903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018650d-128e-4bca-99bf-c59a94a5c6db None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:56.661182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018650d-128e-4bca-99bf-c59a94a5c6db None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:56.661452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018650d-128e-4bca-99bf-c59a94a5c6db None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:56.662873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 724/1473] 10.222.0.42 () {54 vars in 962 bytes} [Wed Sep 14 19:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:56.667837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f06f-e653-4fb5-8bfb-8c1b1c5e9338 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:56.668122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f06f-e653-4fb5-8bfb-8c1b1c5e9338 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:56.668394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f06f-e653-4fb5-8bfb-8c1b1c5e9338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:56.743156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-cae4f06f-e653-4fb5-8bfb-8c1b1c5e9338 None None] MFA Rules not processed for user `4e021a00524341359bc4b5a800a9bbf6`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:56.744355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cae4f06f-e653-4fb5-8bfb-8c1b1c5e9338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:56.765759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:56.820927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 750/1474] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Sep 14 19:48:56 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:56.828571 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018650d-128e-4bca-99bf-c59a94a5c6db None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:48:56.830114 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4018650d-128e-4bca-99bf-c59a94a5c6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:56.931135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018650d-128e-4bca-99bf-c59a94a5c6db None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:48:57.004237 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21002281-e090-4455-9c05-10f179cbd023 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e021a00524341359bc4b5a800a9bbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:48:57.004837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21002281-e090-4455-9c05-10f179cbd023 service glance] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:57.005115 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21002281-e090-4455-9c05-10f179cbd023 service glance] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:57.005386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21002281-e090-4455-9c05-10f179cbd023 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:57.007034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-21002281-e090-4455-9c05-10f179cbd023 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:57.059808 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:48:57.114300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21002281-e090-4455-9c05-10f179cbd023 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:48:57.115752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21002281-e090-4455-9c05-10f179cbd023 service glance] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:48:57.134783 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 725/1475] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:48:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:48:59.345497 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af378f0a-be1f-4b1a-9b2c-45fbf67b501b None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:59.346088 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af378f0a-be1f-4b1a-9b2c-45fbf67b501b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:59.346088 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af378f0a-be1f-4b1a-9b2c-45fbf67b501b None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:59.347409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 751/1476] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:48:59.352641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852dcf5-6803-480e-bd82-a1a1c2297cb0 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:48:59.353165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852dcf5-6803-480e-bd82-a1a1c2297cb0 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:48:59.353165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852dcf5-6803-480e-bd82-a1a1c2297cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:48:59.418839 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e852dcf5-6803-480e-bd82-a1a1c2297cb0 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:48:59.420037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e852dcf5-6803-480e-bd82-a1a1c2297cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:59.514650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 726/1477] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:48:59.793296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af378f0a-be1f-4b1a-9b2c-45fbf67b501b None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:48:59.794875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-af378f0a-be1f-4b1a-9b2c-45fbf67b501b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:48:59.917030 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af378f0a-be1f-4b1a-9b2c-45fbf67b501b None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:48:59.966152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:49:00.008974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e82fa3-d884-40ac-ba40-87333d9822e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:49:00.009567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e82fa3-d884-40ac-ba40-87333d9822e2 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:00.009862 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e82fa3-d884-40ac-ba40-87333d9822e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:00.010128 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e82fa3-d884-40ac-ba40-87333d9822e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:00.010744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e82fa3-d884-40ac-ba40-87333d9822e2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:49:00.012456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-f5e82fa3-d884-40ac-ba40-87333d9822e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:49:00.012456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Sep 14 19:49:00.012456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: Sep 14 19:49:00.013584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e82fa3-d884-40ac-ba40-87333d9822e2 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:49:00.031777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 752/1478] 10.222.0.42 () {56 vars in 1108 bytes} [Wed Sep 14 19:48:59 2022] GET /identity/v3/projects => generated 2553 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:49:02.170887 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09391d-e36d-4b1e-aa82-d1e9d7217144 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:02.171432 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09391d-e36d-4b1e-aa82-d1e9d7217144 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:02.171432 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09391d-e36d-4b1e-aa82-d1e9d7217144 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:02.172751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 727/1479] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:49:02.177973 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aea99b-3cee-4b9e-876c-14746d1448be None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:02.178381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aea99b-3cee-4b9e-876c-14746d1448be None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:02.178518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aea99b-3cee-4b9e-876c-14746d1448be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:02.242964 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-12aea99b-3cee-4b9e-876c-14746d1448be None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:49:02.244212 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-12aea99b-3cee-4b9e-876c-14746d1448be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:02.337962 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 753/1480] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:49:02.632797 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09391d-e36d-4b1e-aa82-d1e9d7217144 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:49:02.634369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1a09391d-e36d-4b1e-aa82-d1e9d7217144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:02.644605 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:49:02.776799 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09391d-e36d-4b1e-aa82-d1e9d7217144 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:49:02.853408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:49:02.887343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d6211d-85cb-4235-8d8d-1ef1c2689aee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:49:02.887942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6211d-85cb-4235-8d8d-1ef1c2689aee service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:02.888215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6211d-85cb-4235-8d8d-1ef1c2689aee service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:02.888507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d6211d-85cb-4235-8d8d-1ef1c2689aee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:02.890117 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a2d6211d-85cb-4235-8d8d-1ef1c2689aee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:02.977035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6211d-85cb-4235-8d8d-1ef1c2689aee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:49:02.978636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d6211d-85cb-4235-8d8d-1ef1c2689aee service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:49:02.995931 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 728/1481] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:49:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:49:05.306320 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade0567-8240-4207-b9fe-1a6464bfdeb1 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:05.306864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade0567-8240-4207-b9fe-1a6464bfdeb1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:05.306864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade0567-8240-4207-b9fe-1a6464bfdeb1 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:05.308185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 754/1482] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:49:05.313504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee53d54-99bd-4170-bb90-4590f9e165fc None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:05.313968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee53d54-99bd-4170-bb90-4590f9e165fc None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:05.314075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee53d54-99bd-4170-bb90-4590f9e165fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:05.381089 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-fee53d54-99bd-4170-bb90-4590f9e165fc None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:49:05.382323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-fee53d54-99bd-4170-bb90-4590f9e165fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:05.427249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:49:05.476368 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 729/1483] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:49:05.762555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade0567-8240-4207-b9fe-1a6464bfdeb1 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:49:05.764087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-eade0567-8240-4207-b9fe-1a6464bfdeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:05.788707 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:49:05.866676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade0567-8240-4207-b9fe-1a6464bfdeb1 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:49:05.939707 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40ad2fb-b0b5-41c3-9a0c-6f8f02577cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:49:05.942920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ad2fb-b0b5-41c3-9a0c-6f8f02577cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:05.942920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ad2fb-b0b5-41c3-9a0c-6f8f02577cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:05.942920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40ad2fb-b0b5-41c3-9a0c-6f8f02577cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:05.943221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e40ad2fb-b0b5-41c3-9a0c-6f8f02577cc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:06.015332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:49:06.048514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ad2fb-b0b5-41c3-9a0c-6f8f02577cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:49:06.050767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40ad2fb-b0b5-41c3-9a0c-6f8f02577cc6 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:49:06.068857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 755/1484] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:49:08.240028 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26b9b8-d5e5-4f0f-a800-d66da8109e5d None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:08.240572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26b9b8-d5e5-4f0f-a800-d66da8109e5d None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:08.240572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26b9b8-d5e5-4f0f-a800-d66da8109e5d None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:08.241937 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 730/1485] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:49:08.247264 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb841881-a2e0-4b41-8b5e-20c07a80b924 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:08.247645 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb841881-a2e0-4b41-8b5e-20c07a80b924 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:08.247842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb841881-a2e0-4b41-8b5e-20c07a80b924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:08.312378 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-fb841881-a2e0-4b41-8b5e-20c07a80b924 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:49:08.313571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fb841881-a2e0-4b41-8b5e-20c07a80b924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:08.406966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 756/1486] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:49:08.696024 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26b9b8-d5e5-4f0f-a800-d66da8109e5d None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:49:08.697558 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0c26b9b8-d5e5-4f0f-a800-d66da8109e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:08.798780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26b9b8-d5e5-4f0f-a800-d66da8109e5d None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:49:08.869574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:49:08.871347 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b19189e-945f-4632-8883-12fda20427bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:49:08.871950 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19189e-945f-4632-8883-12fda20427bb service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:08.872227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19189e-945f-4632-8883-12fda20427bb service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:08.872496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b19189e-945f-4632-8883-12fda20427bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:08.874148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0b19189e-945f-4632-8883-12fda20427bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:08.980758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b19189e-945f-4632-8883-12fda20427bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:49:08.982141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b19189e-945f-4632-8883-12fda20427bb service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:49:09.000250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 731/1487] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:49:11.220209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb07ef0-29c8-4ff2-8812-972d2b6ff4fb None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:11.220783 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb07ef0-29c8-4ff2-8812-972d2b6ff4fb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:11.220783 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb07ef0-29c8-4ff2-8812-972d2b6ff4fb None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:11.222587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 757/1488] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:49:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:49:11.228214 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9d59b-e583-46c8-a51c-dba3656f3afe None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:11.228646 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9d59b-e583-46c8-a51c-dba3656f3afe None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:11.228767 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9d59b-e583-46c8-a51c-dba3656f3afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:11.293940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-99c9d59b-e583-46c8-a51c-dba3656f3afe None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:49:11.295143 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-99c9d59b-e583-46c8-a51c-dba3656f3afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:11.347524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:49:11.388869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 732/1489] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:49:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:49:11.680309 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb07ef0-29c8-4ff2-8812-972d2b6ff4fb None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:49:11.681873 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fbb07ef0-29c8-4ff2-8812-972d2b6ff4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:11.718554 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:49:11.785517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb07ef0-29c8-4ff2-8812-972d2b6ff4fb None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:49:11.866341 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5bd67-f3c0-41d4-ab7b-6b45ebe7f269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:49:11.866940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5bd67-f3c0-41d4-ab7b-6b45ebe7f269 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:11.867217 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5bd67-f3c0-41d4-ab7b-6b45ebe7f269 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:11.867485 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5bd67-f3c0-41d4-ab7b-6b45ebe7f269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:11.869072 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-67c5bd67-f3c0-41d4-ab7b-6b45ebe7f269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:11.958054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:49:11.975893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5bd67-f3c0-41d4-ab7b-6b45ebe7f269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:49:11.977324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5bd67-f3c0-41d4-ab7b-6b45ebe7f269 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:49:11.995769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 758/1490] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:49:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:49:14.480631 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb016d-90ba-4781-9025-e2ddebc60bd8 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:14.481208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb016d-90ba-4781-9025-e2ddebc60bd8 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:14.481208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb016d-90ba-4781-9025-e2ddebc60bd8 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:14.482530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 733/1491] 10.222.0.42 () {54 vars in 927 bytes} [Wed Sep 14 19:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:49:14.488000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f09c1b-1ee9-48bf-b304-f6bb984efa6b None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:14.488406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f09c1b-1ee9-48bf-b304-f6bb984efa6b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:14.488568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f09c1b-1ee9-48bf-b304-f6bb984efa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:14.554675 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-13f09c1b-1ee9-48bf-b304-f6bb984efa6b None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:49:14.555885 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-13f09c1b-1ee9-48bf-b304-f6bb984efa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:14.650268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 759/1492] 10.222.0.42 () {58 vars in 1024 bytes} [Wed Sep 14 19:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:49:14.941310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb016d-90ba-4781-9025-e2ddebc60bd8 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:49:14.942936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-3bbb016d-90ba-4781-9025-e2ddebc60bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:15.044717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb016d-90ba-4781-9025-e2ddebc60bd8 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:49:15.116102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17e6dd2-1bae-481c-9c61-f95c38d76f31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:49:15.116695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e6dd2-1bae-481c-9c61-f95c38d76f31 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:49:15.116968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e6dd2-1bae-481c-9c61-f95c38d76f31 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:49:15.117236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17e6dd2-1bae-481c-9c61-f95c38d76f31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:49:15.118856 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d17e6dd2-1bae-481c-9c61-f95c38d76f31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:49:15.127806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:49:15.226102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e6dd2-1bae-481c-9c61-f95c38d76f31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:49:15.227530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17e6dd2-1bae-481c-9c61-f95c38d76f31 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:49:15.245789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 734/1493] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:49:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:49:24.648504 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=82 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:49:25.243532 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=50 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:52:04.752880 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128902a-de74-4b21-8350-62f72e3ce307 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:04.753487 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128902a-de74-4b21-8350-62f72e3ce307 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:04.753487 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128902a-de74-4b21-8350-62f72e3ce307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:04.833586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5128902a-de74-4b21-8350-62f72e3ce307 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:52:04.834830 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5128902a-de74-4b21-8350-62f72e3ce307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:52:04.937649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 760/1494] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:52:04.946286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7b811-3aeb-49a7-9afb-fa00be8e354f None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:04.946769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7b811-3aeb-49a7-9afb-fa00be8e354f None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:04.946873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a7b811-3aeb-49a7-9afb-fa00be8e354f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:04.966625 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-a4a7b811-3aeb-49a7-9afb-fa00be8e354f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:52:04.967803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a4a7b811-3aeb-49a7-9afb-fa00be8e354f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:52:04.976029 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 735/1495] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:52:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:52:04.980311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128902a-de74-4b21-8350-62f72e3ce307 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:52:04.989572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128902a-de74-4b21-8350-62f72e3ce307 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:52:04.994172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adcb5f-eaab-43e7-95d7-8830ccbdbcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:52:04.994759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adcb5f-eaab-43e7-95d7-8830ccbdbcc0 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:04.995033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adcb5f-eaab-43e7-95d7-8830ccbdbcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:04.995300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57adcb5f-eaab-43e7-95d7-8830ccbdbcc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:04.995928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57adcb5f-eaab-43e7-95d7-8830ccbdbcc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:52:04.996221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57adcb5f-eaab-43e7-95d7-8830ccbdbcc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:52:04.997223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-57adcb5f-eaab-43e7-95d7-8830ccbdbcc0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:52:04.997223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Sep 14 19:52:04.997223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: Sep 14 19:52:04.998142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57adcb5f-eaab-43e7-95d7-8830ccbdbcc0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:52:05.009840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 761/1496] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:52:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:52:05.017751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7b811-3aeb-49a7-9afb-fa00be8e354f None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:52:05.026672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a7b811-3aeb-49a7-9afb-fa00be8e354f None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:52:05.031169 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e816b8-2a29-4439-a4b8-33125f644859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:52:05.031746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e816b8-2a29-4439-a4b8-33125f644859 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:05.032021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e816b8-2a29-4439-a4b8-33125f644859 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:05.032289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e816b8-2a29-4439-a4b8-33125f644859 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:05.032911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e816b8-2a29-4439-a4b8-33125f644859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:52:05.034678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: WARNING py.warnings [None req-35e816b8-2a29-4439-a4b8-33125f644859 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:52:05.034678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: warnings.warn(msg) Sep 14 19:52:05.034678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: Sep 14 19:52:05.035861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e816b8-2a29-4439-a4b8-33125f644859 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:52:05.135242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 736/1497] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:52:05 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:52:05.139353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adcb5f-eaab-43e7-95d7-8830ccbdbcc0 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:52:05.150004 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57adcb5f-eaab-43e7-95d7-8830ccbdbcc0 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:52:05.199041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:52:05.205740 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ba1df-8639-4836-b8c2-fa8492e63f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:52:05.206361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ba1df-8639-4836-b8c2-fa8492e63f40 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:05.206636 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ba1df-8639-4836-b8c2-fa8492e63f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:05.206919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ba1df-8639-4836-b8c2-fa8492e63f40 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:05.207577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ba1df-8639-4836-b8c2-fa8492e63f40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:52:05.208573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-353ba1df-8639-4836-b8c2-fa8492e63f40 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:52:05.208573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Sep 14 19:52:05.208573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: Sep 14 19:52:05.209765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ba1df-8639-4836-b8c2-fa8492e63f40 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:52:05.316967 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 762/1498] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:52:05 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:52:05.321861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e816b8-2a29-4439-a4b8-33125f644859 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:52:05.331935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e816b8-2a29-4439-a4b8-33125f644859 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:52:05.336466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421a1c7-7e8b-4652-8e71-1fe7cd506b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:52:05.337048 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421a1c7-7e8b-4652-8e71-1fe7cd506b08 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:05.337320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421a1c7-7e8b-4652-8e71-1fe7cd506b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:05.337587 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421a1c7-7e8b-4652-8e71-1fe7cd506b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:05.338244 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421a1c7-7e8b-4652-8e71-1fe7cd506b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:52:05.339237 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: WARNING py.warnings [None req-f421a1c7-7e8b-4652-8e71-1fe7cd506b08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:52:05.339237 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: warnings.warn(msg) Sep 14 19:52:05.339237 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: Sep 14 19:52:05.340116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421a1c7-7e8b-4652-8e71-1fe7cd506b08 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:52:05.352906 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 737/1499] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:52:05.356676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ba1df-8639-4836-b8c2-fa8492e63f40 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:52:05.366900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ba1df-8639-4836-b8c2-fa8492e63f40 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:52:05.371430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb3752-9e4a-4e32-8512-3059465cecaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:52:05.372014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb3752-9e4a-4e32-8512-3059465cecaa admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:05.372295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb3752-9e4a-4e32-8512-3059465cecaa admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:05.372563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb3752-9e4a-4e32-8512-3059465cecaa admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:05.373184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb3752-9e4a-4e32-8512-3059465cecaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:52:05.374206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-e3fb3752-9e4a-4e32-8512-3059465cecaa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:52:05.374206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Sep 14 19:52:05.374206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: Sep 14 19:52:05.375079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb3752-9e4a-4e32-8512-3059465cecaa admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:52:05.388646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 763/1500] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:52:05.392678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421a1c7-7e8b-4652-8e71-1fe7cd506b08 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:52:05.402423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421a1c7-7e8b-4652-8e71-1fe7cd506b08 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:52:05.406983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a134f740-d701-484a-9f32-8aa1cb16518a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:52:05.407620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134f740-d701-484a-9f32-8aa1cb16518a admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:05.407893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134f740-d701-484a-9f32-8aa1cb16518a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:05.408162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a134f740-d701-484a-9f32-8aa1cb16518a admin admin] PATH_INFO: `/v3/projects/99fe2f76e5bb47a69054b67ae1d3e098/users/1cbb217bfe6f41d5b16ad67930e84b64/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:05.423619 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a134f740-d701-484a-9f32-8aa1cb16518a admin admin] RBAC: Authorizing `identity:create_grant(project_id=99fe2f76e5bb47a69054b67ae1d3e098, user_id=1cbb217bfe6f41d5b16ad67930e84b64, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:52:05.424620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: WARNING py.warnings [None req-a134f740-d701-484a-9f32-8aa1cb16518a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:52:05.424620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: warnings.warn(msg) Sep 14 19:52:05.424620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: Sep 14 19:52:05.426635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a134f740-d701-484a-9f32-8aa1cb16518a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:52:05.511899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 738/1501] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:52:05 2022] PUT /identity/v3/projects/99fe2f76e5bb47a69054b67ae1d3e098/users/1cbb217bfe6f41d5b16ad67930e84b64/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:52:05.523829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f85755-2d5b-4c0b-9433-914683ebaa4b None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:05.524933 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f85755-2d5b-4c0b-9433-914683ebaa4b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:05.524933 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f85755-2d5b-4c0b-9433-914683ebaa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:05.552465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f8f85755-2d5b-4c0b-9433-914683ebaa4b None None] MFA Rules not processed for user `1cbb217bfe6f41d5b16ad67930e84b64`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:52:05.553659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f8f85755-2d5b-4c0b-9433-914683ebaa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:52:05.615760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 764/1502] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:52:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:52:05.629259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318b120-359a-4e33-8752-7dc59ee2bb3f None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:05.629559 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318b120-359a-4e33-8752-7dc59ee2bb3f None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:05.629843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9318b120-359a-4e33-8752-7dc59ee2bb3f None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:05.631163 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 739/1503] 10.222.0.42 () {54 vars in 964 bytes} [Wed Sep 14 19:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:52:05.635733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6832f3-5992-4f10-80f5-a93e6933530b None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:05.636017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6832f3-5992-4f10-80f5-a93e6933530b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:05.636295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6832f3-5992-4f10-80f5-a93e6933530b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:05.694041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-ff6832f3-5992-4f10-80f5-a93e6933530b None None] MFA Rules not processed for user `8a2c9eec08b142138569043e3b34c6a8`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:52:05.695238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ff6832f3-5992-4f10-80f5-a93e6933530b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:52:05.767841 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 765/1504] 10.222.0.42 () {58 vars in 1061 bytes} [Wed Sep 14 19:52:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:52:05.775492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318b120-359a-4e33-8752-7dc59ee2bb3f None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:52:05.785789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9318b120-359a-4e33-8752-7dc59ee2bb3f None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:52:05.789826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d56c5-d119-4ed4-9e37-8f9268f24983 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:52:05.790404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d56c5-d119-4ed4-9e37-8f9268f24983 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:05.790678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d56c5-d119-4ed4-9e37-8f9268f24983 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:05.790946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d56c5-d119-4ed4-9e37-8f9268f24983 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:05.798846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d56c5-d119-4ed4-9e37-8f9268f24983 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:52:05.800239 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d56c5-d119-4ed4-9e37-8f9268f24983 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:52:05.809321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 740/1505] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:52:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:52:05.854209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f94b10-a58d-4203-b5e0-decef40b2077 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:05.854598 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f94b10-a58d-4203-b5e0-decef40b2077 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:05.854904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f94b10-a58d-4203-b5e0-decef40b2077 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:05.856172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 766/1506] 10.222.0.42 () {52 vars in 830 bytes} [Wed Sep 14 19:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:52:05.859896 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d56c5-d119-4ed4-9e37-8f9268f24983 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:52:05.921502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d56c5-d119-4ed4-9e37-8f9268f24983 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:52:05.926086 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:52:05.926701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:05.926975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:05.927244 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] PATH_INFO: `/v3/users/1cbb217bfe6f41d5b16ad67930e84b64` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:05.928352 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cbb217bfe6f41d5b16ad67930e84b64)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:52:05.929355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: WARNING py.warnings [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:52:05.929355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: warnings.warn(msg) Sep 14 19:52:05.929355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: Sep 14 19:52:05.930577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:52:06.173054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cbb217bfe6f41d5b16ad67930e84b64'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:52:06.196320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cbb217bfe6f41d5b16ad67930e84b64'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:52:06.237686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cbb217bfe6f41d5b16ad67930e84b64'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:52:06.238331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cbb217bfe6f41d5b16ad67930e84b64'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:52:06.248675 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:52:06.253041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cbb217bfe6f41d5b16ad67930e84b64'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:52:06.259454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 741/1507] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:52:05 2022] DELETE /identity/v3/users/1cbb217bfe6f41d5b16ad67930e84b64 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:52:06.263822 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e1cb3-b570-4ef2-852b-8acd1b94f6dd None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:06.264284 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e1cb3-b570-4ef2-852b-8acd1b94f6dd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:06.264564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072e1cb3-b570-4ef2-852b-8acd1b94f6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:06.283938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-072e1cb3-b570-4ef2-852b-8acd1b94f6dd None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:52:06.285136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-072e1cb3-b570-4ef2-852b-8acd1b94f6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:52:06.294205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:52:06.343739 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 767/1508] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:52:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:52:06.358847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:52:06.358847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:52:06.430611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a22c7-d3ec-49a0-a801-f84a1e988782 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:52:06.434181 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793693e-b4dc-4e97-a16d-f56c3e8b237a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:52:06.434778 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793693e-b4dc-4e97-a16d-f56c3e8b237a service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:06.435052 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793693e-b4dc-4e97-a16d-f56c3e8b237a service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:06.435322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793693e-b4dc-4e97-a16d-f56c3e8b237a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:06.444182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793693e-b4dc-4e97-a16d-f56c3e8b237a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:52:06.446137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793693e-b4dc-4e97-a16d-f56c3e8b237a service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:52:06.455211 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 742/1509] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:52:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:52:06.960906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e1cb3-b570-4ef2-852b-8acd1b94f6dd None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:52:06.962454 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-072e1cb3-b570-4ef2-852b-8acd1b94f6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:52:06.974189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072e1cb3-b570-4ef2-852b-8acd1b94f6dd None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:52:06.979543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead93824-7950-41c3-b201-b02ab7955d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:52:06.980031 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead93824-7950-41c3-b201-b02ab7955d5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:52:06.980326 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead93824-7950-41c3-b201-b02ab7955d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:52:06.980592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead93824-7950-41c3-b201-b02ab7955d5f admin admin] PATH_INFO: `/v3/projects/99fe2f76e5bb47a69054b67ae1d3e098` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:52:06.981816 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead93824-7950-41c3-b201-b02ab7955d5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=99fe2f76e5bb47a69054b67ae1d3e098)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:52:06.982813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-ead93824-7950-41c3-b201-b02ab7955d5f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:52:06.982813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Sep 14 19:52:06.982813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: Sep 14 19:52:06.985663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead93824-7950-41c3-b201-b02ab7955d5f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:52:07.122530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ead93824-7950-41c3-b201-b02ab7955d5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99fe2f76e5bb47a69054b67ae1d3e098', 'request_id': 'req-ead93824-7950-41c3-b201-b02ab7955d5f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:52:07.220941 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ead93824-7950-41c3-b201-b02ab7955d5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99fe2f76e5bb47a69054b67ae1d3e098', 'request_id': 'req-ead93824-7950-41c3-b201-b02ab7955d5f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:52:07.236340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ead93824-7950-41c3-b201-b02ab7955d5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99fe2f76e5bb47a69054b67ae1d3e098', 'request_id': 'req-ead93824-7950-41c3-b201-b02ab7955d5f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:52:07.247534 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ead93824-7950-41c3-b201-b02ab7955d5f admin admin] The token cache is being invalidate because project 99fe2f76e5bb47a69054b67ae1d3e098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:52:07.247816 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ead93824-7950-41c3-b201-b02ab7955d5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:52:07.248567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ead93824-7950-41c3-b201-b02ab7955d5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:52:07.250046 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 768/1510] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:52:06 2022] DELETE /identity/v3/projects/99fe2f76e5bb47a69054b67ae1d3e098 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:52:16.451599 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=38 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:52:17.244724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:53:36.417790 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88751279-f559-42e6-b9cf-bae8af95a723 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:36.418913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88751279-f559-42e6-b9cf-bae8af95a723 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:36.418913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88751279-f559-42e6-b9cf-bae8af95a723 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:36.420199 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 743/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 19:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:53:36.425146 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0b623-8797-4fdb-9c37-0819f05a906e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:36.425690 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0b623-8797-4fdb-9c37-0819f05a906e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:36.425989 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0b623-8797-4fdb-9c37-0819f05a906e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:36.447011 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-41f0b623-8797-4fdb-9c37-0819f05a906e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:53:36.448203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-41f0b623-8797-4fdb-9c37-0819f05a906e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:53:36.505060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 769/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 19:53:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:53:36.661318 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88751279-f559-42e6-b9cf-bae8af95a723 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:53:36.663064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-88751279-f559-42e6-b9cf-bae8af95a723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:53:36.765293 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88751279-f559-42e6-b9cf-bae8af95a723 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:53:36.770797 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9337c-dca5-43e1-b3e4-45cd349ebf36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:53:36.771401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9337c-dca5-43e1-b3e4-45cd349ebf36 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:36.771683 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9337c-dca5-43e1-b3e4-45cd349ebf36 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:36.771955 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9337c-dca5-43e1-b3e4-45cd349ebf36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:36.780467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9337c-dca5-43e1-b3e4-45cd349ebf36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:53:36.782077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9337c-dca5-43e1-b3e4-45cd349ebf36 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:53:36.791293 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 744/1513] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:53:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:53:37.864428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c9a3a-f0be-4100-88f3-de8733821f83 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:37.864939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c9a3a-f0be-4100-88f3-de8733821f83 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:37.865033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027c9a3a-f0be-4100-88f3-de8733821f83 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:37.866328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 770/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 19:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:53:37.873580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cf088-ac98-46d9-ab3e-07ac8ae89e27 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:37.874073 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cf088-ac98-46d9-ab3e-07ac8ae89e27 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:37.874184 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104cf088-ac98-46d9-ab3e-07ac8ae89e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:37.893657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-104cf088-ac98-46d9-ab3e-07ac8ae89e27 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:53:37.894907 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-104cf088-ac98-46d9-ab3e-07ac8ae89e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:53:37.903076 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 745/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 19:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:53:38.057604 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c9a3a-f0be-4100-88f3-de8733821f83 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:53:38.071838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027c9a3a-f0be-4100-88f3-de8733821f83 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:53:38.077121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a959cd-1df9-40c8-8f29-15e804892650 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:53:38.077705 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a959cd-1df9-40c8-8f29-15e804892650 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:38.078011 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a959cd-1df9-40c8-8f29-15e804892650 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:38.078278 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a959cd-1df9-40c8-8f29-15e804892650 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:38.086061 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a959cd-1df9-40c8-8f29-15e804892650 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:53:38.087617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a959cd-1df9-40c8-8f29-15e804892650 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:53:38.097112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 771/1516] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:53:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:53:38.375272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8179-2ccb-4c7f-a936-638cd6758067 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:38.375488 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8179-2ccb-4c7f-a936-638cd6758067 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:38.375734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b8179-2ccb-4c7f-a936-638cd6758067 None None] PATH_INFO: `/v3` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:38.377022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 746/1517] 10.222.0.26 () {54 vars in 925 bytes} [Wed Sep 14 19:53:38 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:53:38.383963 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f568386-587c-404f-9f15-97ee8bd8d136 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:38.384243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f568386-587c-404f-9f15-97ee8bd8d136 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:38.384511 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f568386-587c-404f-9f15-97ee8bd8d136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:38.433154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5f568386-587c-404f-9f15-97ee8bd8d136 None None] MFA Rules not processed for user `03bc280d0395479981b7c7b08c96001b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:53:38.434419 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5f568386-587c-404f-9f15-97ee8bd8d136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:53:38.504286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 772/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Sep 14 19:53:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:53:42.407812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae549bd7-4bf5-4986-a0e6-c9e5bea2e38f None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:42.408301 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae549bd7-4bf5-4986-a0e6-c9e5bea2e38f None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:42.408457 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae549bd7-4bf5-4986-a0e6-c9e5bea2e38f None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:42.409877 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 747/1519] 10.222.0.42 () {54 vars in 961 bytes} [Wed Sep 14 19:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:53:42.414783 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5274d8-914c-494f-9a18-ebceac109d06 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:42.415228 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5274d8-914c-494f-9a18-ebceac109d06 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:42.415358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5274d8-914c-494f-9a18-ebceac109d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:42.435133 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-da5274d8-914c-494f-9a18-ebceac109d06 None None] MFA Rules not processed for user `03bc280d0395479981b7c7b08c96001b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:53:42.436326 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-da5274d8-914c-494f-9a18-ebceac109d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:53:42.445064 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 773/1520] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Sep 14 19:53:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:53:42.453191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae549bd7-4bf5-4986-a0e6-c9e5bea2e38f None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:53:42.466292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae549bd7-4bf5-4986-a0e6-c9e5bea2e38f None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:53:42.471133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01dfffa-136d-486f-83b1-9aa372f3bb1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03bc280d0395479981b7c7b08c96001b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:53:42.471715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01dfffa-136d-486f-83b1-9aa372f3bb1a service placement] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:42.471988 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01dfffa-136d-486f-83b1-9aa372f3bb1a service placement] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:42.472256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01dfffa-136d-486f-83b1-9aa372f3bb1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:42.480251 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01dfffa-136d-486f-83b1-9aa372f3bb1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:53:42.482486 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01dfffa-136d-486f-83b1-9aa372f3bb1a service placement] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:53:42.491033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 748/1521] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:53:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:53:49.567885 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098e11a-1b98-489d-acbb-582ff7b7dab2 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:49.568467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098e11a-1b98-489d-acbb-582ff7b7dab2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:49.568467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098e11a-1b98-489d-acbb-582ff7b7dab2 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:49.569763 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 774/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 19:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:53:49.575772 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4f69f-d6b6-4163-93a7-9bdcb17b45ea None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:49.576259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4f69f-d6b6-4163-93a7-9bdcb17b45ea None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:49.576358 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc4f69f-d6b6-4163-93a7-9bdcb17b45ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:49.595368 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-6bc4f69f-d6b6-4163-93a7-9bdcb17b45ea None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:53:49.596559 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6bc4f69f-d6b6-4163-93a7-9bdcb17b45ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:53:49.604330 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 749/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 19:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:53:49.758574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098e11a-1b98-489d-acbb-582ff7b7dab2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:53:49.772781 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098e11a-1b98-489d-acbb-582ff7b7dab2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:53:49.778243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e3a11-224b-4612-8920-0018663b11fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:53:49.778856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e3a11-224b-4612-8920-0018663b11fe service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:49.779128 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e3a11-224b-4612-8920-0018663b11fe service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:49.779399 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e3a11-224b-4612-8920-0018663b11fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:49.787744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e3a11-224b-4612-8920-0018663b11fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:53:49.789330 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e3a11-224b-4612-8920-0018663b11fe service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:53:49.799424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 775/1524] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:53:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:53:50.866073 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b249b-695f-4fcc-989d-b993dd06b1cb None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:50.866620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b249b-695f-4fcc-989d-b993dd06b1cb None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:50.866620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b249b-695f-4fcc-989d-b993dd06b1cb None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:50.867983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 750/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 19:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:53:50.874858 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2b418-369c-45d6-aa62-9494a878af30 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:50.875286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2b418-369c-45d6-aa62-9494a878af30 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:50.875393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2b418-369c-45d6-aa62-9494a878af30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:50.894937 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1ea2b418-369c-45d6-aa62-9494a878af30 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:53:50.896155 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1ea2b418-369c-45d6-aa62-9494a878af30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:53:50.903729 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 776/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 19:53:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:53:51.054725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b249b-695f-4fcc-989d-b993dd06b1cb None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:53:51.069632 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b249b-695f-4fcc-989d-b993dd06b1cb None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:53:51.075027 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07183ee5-1518-488e-a1de-dec8a664b42f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:53:51.075608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07183ee5-1518-488e-a1de-dec8a664b42f service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:51.075881 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07183ee5-1518-488e-a1de-dec8a664b42f service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:51.076149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07183ee5-1518-488e-a1de-dec8a664b42f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:51.088420 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07183ee5-1518-488e-a1de-dec8a664b42f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:53:51.090005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07183ee5-1518-488e-a1de-dec8a664b42f service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:53:51.099275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 751/1527] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:53:55.241030 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f105b-1522-4af1-952c-973372cb0d87 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:55.241544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f105b-1522-4af1-952c-973372cb0d87 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:55.241544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89f105b-1522-4af1-952c-973372cb0d87 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:55.243414 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 777/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:53:55.249028 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92271829-37e0-486b-a9ea-59b87da306e1 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:55.249445 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92271829-37e0-486b-a9ea-59b87da306e1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:55.249563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92271829-37e0-486b-a9ea-59b87da306e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:55.269305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-92271829-37e0-486b-a9ea-59b87da306e1 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:53:55.270538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-92271829-37e0-486b-a9ea-59b87da306e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:53:55.278369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 752/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:53:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:53:55.567581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f105b-1522-4af1-952c-973372cb0d87 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:53:55.569268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c89f105b-1522-4af1-952c-973372cb0d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:53:55.597879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:53:55.625728 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89f105b-1522-4af1-952c-973372cb0d87 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:53:55.629236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb3f46-31c5-409e-b29c-925e8ed55617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:53:55.629861 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb3f46-31c5-409e-b29c-925e8ed55617 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:55.630142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb3f46-31c5-409e-b29c-925e8ed55617 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:55.630412 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb3f46-31c5-409e-b29c-925e8ed55617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:55.639551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb3f46-31c5-409e-b29c-925e8ed55617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:53:55.640960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb3f46-31c5-409e-b29c-925e8ed55617 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:53:55.650131 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 778/1530] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:53:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:53:57.789714 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e1468-c3b7-4d8b-95e8-71289ef9b4df None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:57.790341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e1468-c3b7-4d8b-95e8-71289ef9b4df None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:57.790341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e1468-c3b7-4d8b-95e8-71289ef9b4df None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:57.791647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 753/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:53:57.797716 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbe1b3-e3d3-47cd-8925-ea37875be368 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:57.798206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbe1b3-e3d3-47cd-8925-ea37875be368 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:57.798347 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbbe1b3-e3d3-47cd-8925-ea37875be368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:57.818600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7fbbe1b3-e3d3-47cd-8925-ea37875be368 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:53:57.819795 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7fbbe1b3-e3d3-47cd-8925-ea37875be368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:53:57.827551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 779/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:53:58.112254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e1468-c3b7-4d8b-95e8-71289ef9b4df None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:53:58.123481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e1468-c3b7-4d8b-95e8-71289ef9b4df None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:53:58.126984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e9914-552a-4d83-a428-384644b8330b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:53:58.127572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e9914-552a-4d83-a428-384644b8330b service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:53:58.127847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e9914-552a-4d83-a428-384644b8330b service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:53:58.128115 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e9914-552a-4d83-a428-384644b8330b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:53:58.136860 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e9914-552a-4d83-a428-384644b8330b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:53:58.138339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e9914-552a-4d83-a428-384644b8330b service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:53:58.147436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 754/1533] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:53:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:54:01.954921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c5cbf-6064-4b8d-ac05-8d5626e38d75 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:54:01.955371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c5cbf-6064-4b8d-ac05-8d5626e38d75 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:54:01.955467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c5cbf-6064-4b8d-ac05-8d5626e38d75 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:54:01.956770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 780/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:54:01.961873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988eb821-21f1-4244-8303-2026d7bf6d64 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:54:01.962307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988eb821-21f1-4244-8303-2026d7bf6d64 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:54:01.962411 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988eb821-21f1-4244-8303-2026d7bf6d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:54:01.984227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-988eb821-21f1-4244-8303-2026d7bf6d64 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:54:01.985404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-988eb821-21f1-4244-8303-2026d7bf6d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:54:01.993020 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 755/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:54:02.294523 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c5cbf-6064-4b8d-ac05-8d5626e38d75 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:54:02.297516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-952c5cbf-6064-4b8d-ac05-8d5626e38d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:54:02.352862 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c5cbf-6064-4b8d-ac05-8d5626e38d75 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:54:02.356473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08237d4-8945-4ab6-a0e2-4366932f16db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:54:02.357077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08237d4-8945-4ab6-a0e2-4366932f16db service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:54:02.357352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08237d4-8945-4ab6-a0e2-4366932f16db service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:54:02.357623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08237d4-8945-4ab6-a0e2-4366932f16db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:54:02.366353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08237d4-8945-4ab6-a0e2-4366932f16db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:54:02.367764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08237d4-8945-4ab6-a0e2-4366932f16db service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:54:02.376918 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 781/1536] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:54:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:54:04.510897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa30e4f-2626-41b6-a407-aff491341f3a None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:54:04.511506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa30e4f-2626-41b6-a407-aff491341f3a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:54:04.511506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa30e4f-2626-41b6-a407-aff491341f3a None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:54:04.513328 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 756/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 19:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:54:04.518604 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ca18-5e3c-49fd-aba2-eb33d1abb670 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:54:04.519029 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ca18-5e3c-49fd-aba2-eb33d1abb670 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:54:04.519157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e8ca18-5e3c-49fd-aba2-eb33d1abb670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:54:04.538394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-17e8ca18-5e3c-49fd-aba2-eb33d1abb670 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:54:04.539587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-17e8ca18-5e3c-49fd-aba2-eb33d1abb670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:54:04.547989 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 782/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 19:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:54:04.846942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa30e4f-2626-41b6-a407-aff491341f3a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:54:04.848579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-afa30e4f-2626-41b6-a407-aff491341f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:54:04.861912 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa30e4f-2626-41b6-a407-aff491341f3a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:54:04.865365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95be722-ce11-4a68-be75-a44dadd2827c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:54:04.865994 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95be722-ce11-4a68-be75-a44dadd2827c service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:54:04.866270 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95be722-ce11-4a68-be75-a44dadd2827c service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:54:04.866541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95be722-ce11-4a68-be75-a44dadd2827c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:54:04.876180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95be722-ce11-4a68-be75-a44dadd2827c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:54:04.877595 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95be722-ce11-4a68-be75-a44dadd2827c service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:54:04.886830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 757/1539] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:54:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:54:14.533436 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=47 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:54:14.882671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=82 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:31.035626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87b36a-9bd7-4a05-a3e7-ef10200124f0 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.036339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87b36a-9bd7-4a05-a3e7-ef10200124f0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.036339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87b36a-9bd7-4a05-a3e7-ef10200124f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.063827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-9a87b36a-9bd7-4a05-a3e7-ef10200124f0 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:31.066606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9a87b36a-9bd7-4a05-a3e7-ef10200124f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:31.083647 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 783/1540] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:31.094099 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35eb236-dbd9-496f-b49e-f9f1f641fbea None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.094716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35eb236-dbd9-496f-b49e-f9f1f641fbea None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.095081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35eb236-dbd9-496f-b49e-f9f1f641fbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.271130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-a35eb236-dbd9-496f-b49e-f9f1f641fbea None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:31.273038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a35eb236-dbd9-496f-b49e-f9f1f641fbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:31.289145 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 758/1541] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:55:31.293336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87b36a-9bd7-4a05-a3e7-ef10200124f0 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:31.304765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87b36a-9bd7-4a05-a3e7-ef10200124f0 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:31.317557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ccd69-814f-4121-9fae-af1bb801d9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:31.318203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ccd69-814f-4121-9fae-af1bb801d9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.318477 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ccd69-814f-4121-9fae-af1bb801d9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.318738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ccd69-814f-4121-9fae-af1bb801d9b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.319383 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ccd69-814f-4121-9fae-af1bb801d9b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:55:31.319677 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ccd69-814f-4121-9fae-af1bb801d9b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:31.321317 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ccd69-814f-4121-9fae-af1bb801d9b7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:31.338428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8a8ef-c921-4aed-9c55-ad03e82edb29 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.339063 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8a8ef-c921-4aed-9c55-ad03e82edb29 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.339545 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8a8ef-c921-4aed-9c55-ad03e82edb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.341891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 784/1542] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:55:31.350357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ccd69-814f-4121-9fae-af1bb801d9b7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:31.368199 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ccd69-814f-4121-9fae-af1bb801d9b7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:31.375618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e06e7-da32-42c8-a62d-ed437d6d1ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:31.380078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e06e7-da32-42c8-a62d-ed437d6d1ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.380078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e06e7-da32-42c8-a62d-ed437d6d1ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.380078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e06e7-da32-42c8-a62d-ed437d6d1ca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.380078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e06e7-da32-42c8-a62d-ed437d6d1ca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:31.380078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-9f4e06e7-da32-42c8-a62d-ed437d6d1ca9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:55:31.380078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Sep 14 19:55:31.380078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: Sep 14 19:55:31.380674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4e06e7-da32-42c8-a62d-ed437d6d1ca9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:31.395011 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b3d8a8ef-c921-4aed-9c55-ad03e82edb29 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:31.396689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b3d8a8ef-c921-4aed-9c55-ad03e82edb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:31.410835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 759/1543] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:55:31.419903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663de36-a521-42cf-912a-b549d7f5e436 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.420502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663de36-a521-42cf-912a-b549d7f5e436 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.420996 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d663de36-a521-42cf-912a-b549d7f5e436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.449150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-d663de36-a521-42cf-912a-b549d7f5e436 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:31.450771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d663de36-a521-42cf-912a-b549d7f5e436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:31.464286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 760/1544] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:55:31.469893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663de36-a521-42cf-912a-b549d7f5e436 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:31.482117 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d663de36-a521-42cf-912a-b549d7f5e436 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:31.489388 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec2f77-464b-4f46-830c-3576af06e65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:31.490034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec2f77-464b-4f46-830c-3576af06e65b admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.490314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec2f77-464b-4f46-830c-3576af06e65b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.490635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec2f77-464b-4f46-830c-3576af06e65b admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.491217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec2f77-464b-4f46-830c-3576af06e65b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:55:31.491516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec2f77-464b-4f46-830c-3576af06e65b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:31.492531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: WARNING py.warnings [None req-ceec2f77-464b-4f46-830c-3576af06e65b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:55:31.492531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: warnings.warn(msg) Sep 14 19:55:31.492531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: Sep 14 19:55:31.493412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec2f77-464b-4f46-830c-3576af06e65b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:31.512140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 761/1545] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:55:31.513599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 785/1546] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:55:31.516431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbf00e-b02f-4641-9627-16f6ad12881a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.516714 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbf00e-b02f-4641-9627-16f6ad12881a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.516983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbf00e-b02f-4641-9627-16f6ad12881a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.520955 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e06e7-da32-42c8-a62d-ed437d6d1ca9 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:31.549007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-06fbf00e-b02f-4641-9627-16f6ad12881a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:31.550259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-06fbf00e-b02f-4641-9627-16f6ad12881a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:31.612292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e06e7-da32-42c8-a62d-ed437d6d1ca9 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:31.617111 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0652dc93-4d83-4e06-9252-1a7495aec500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:31.617715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652dc93-4d83-4e06-9252-1a7495aec500 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.618013 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652dc93-4d83-4e06-9252-1a7495aec500 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.618284 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0652dc93-4d83-4e06-9252-1a7495aec500 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.618907 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652dc93-4d83-4e06-9252-1a7495aec500 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:31.620825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0652dc93-4d83-4e06-9252-1a7495aec500 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:31.624302 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 762/1547] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:31.625152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbf00e-b02f-4641-9627-16f6ad12881a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:31.634555 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbf00e-b02f-4641-9627-16f6ad12881a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:31.639139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1adca-b3c2-43be-b8b0-1f9dc30c147a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:31.643224 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1adca-b3c2-43be-b8b0-1f9dc30c147a admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.643224 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1adca-b3c2-43be-b8b0-1f9dc30c147a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.643224 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1adca-b3c2-43be-b8b0-1f9dc30c147a admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.643224 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1adca-b3c2-43be-b8b0-1f9dc30c147a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:31.643605 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1adca-b3c2-43be-b8b0-1f9dc30c147a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:31.725588 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 786/1548] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/users => generated 289 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:55:31.726910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416543a-64f2-4165-8a09-1b12e942d08c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.727190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416543a-64f2-4165-8a09-1b12e942d08c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.730000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416543a-64f2-4165-8a09-1b12e942d08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.746565 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d416543a-64f2-4165-8a09-1b12e942d08c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:31.747728 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d416543a-64f2-4165-8a09-1b12e942d08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:31.755455 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 787/1549] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:31.756562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26982a-4403-4781-8457-9c5fd528cb1a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.756834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26982a-4403-4781-8457-9c5fd528cb1a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.757100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26982a-4403-4781-8457-9c5fd528cb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.772501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 763/1550] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/projects => generated 391 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:31.773826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a1fe3-5e10-422b-a42e-160e215e8ba9 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.774116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a1fe3-5e10-422b-a42e-160e215e8ba9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.774379 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1a1fe3-5e10-422b-a42e-160e215e8ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.777314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bd26982a-4403-4781-8457-9c5fd528cb1a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:31.778532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bd26982a-4403-4781-8457-9c5fd528cb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:31.794197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-6a1a1fe3-5e10-422b-a42e-160e215e8ba9 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:31.795377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6a1a1fe3-5e10-422b-a42e-160e215e8ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:31.839323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 788/1551] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:31.840644 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c408e-baf6-415a-a963-04a12fc05f4f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.840929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c408e-baf6-415a-a963-04a12fc05f4f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.841196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c408e-baf6-415a-a963-04a12fc05f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.863623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a81c408e-baf6-415a-a963-04a12fc05f4f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:31.864827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a81c408e-baf6-415a-a963-04a12fc05f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:31.866322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 764/1552] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:31.867624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a6d12-61eb-484e-ac10-84feb14e4db5 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.868364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a6d12-61eb-484e-ac10-84feb14e4db5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.868764 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35a6d12-61eb-484e-ac10-84feb14e4db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.881675 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 789/1553] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:31.882961 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee2c3c-69b5-43c4-a4a5-45c70363638c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.883240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee2c3c-69b5-43c4-a4a5-45c70363638c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.883509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee2c3c-69b5-43c4-a4a5-45c70363638c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.889390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-f35a6d12-61eb-484e-ac10-84feb14e4db5 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:31.890700 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-f35a6d12-61eb-484e-ac10-84feb14e4db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:31.898597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 765/1554] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:31.899445 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a6d12-61eb-484e-ac10-84feb14e4db5 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:31.903760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-dcee2c3c-69b5-43c4-a4a5-45c70363638c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:31.905019 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-dcee2c3c-69b5-43c4-a4a5-45c70363638c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:31.909583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35a6d12-61eb-484e-ac10-84feb14e4db5 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:31.913531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 790/1555] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:31.914301 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59676e03-c483-4a7e-a224-578401072196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:31.914732 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836388a9-90ad-45ee-9d1e-d125049dab6e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.915012 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836388a9-90ad-45ee-9d1e-d125049dab6e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.915286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836388a9-90ad-45ee-9d1e-d125049dab6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.915682 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59676e03-c483-4a7e-a224-578401072196 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.916118 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59676e03-c483-4a7e-a224-578401072196 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.916552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59676e03-c483-4a7e-a224-578401072196 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.917529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59676e03-c483-4a7e-a224-578401072196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:31.919146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59676e03-c483-4a7e-a224-578401072196 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:31.929172 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 766/1556] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:31.930162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59676e03-c483-4a7e-a224-578401072196 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:31.935274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-836388a9-90ad-45ee-9d1e-d125049dab6e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:31.936469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-836388a9-90ad-45ee-9d1e-d125049dab6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:31.940250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59676e03-c483-4a7e-a224-578401072196 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:31.944699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593bc69-30fa-4a96-b3d7-13a1442da6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:31.945104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 791/1557] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:31.945291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593bc69-30fa-4a96-b3d7-13a1442da6aa admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.945537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593bc69-30fa-4a96-b3d7-13a1442da6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.945916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593bc69-30fa-4a96-b3d7-13a1442da6aa admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.946306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c3150-6e9e-429c-b637-ad9f7cdb94ea None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.946525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593bc69-30fa-4a96-b3d7-13a1442da6aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:31.946632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c3150-6e9e-429c-b637-ad9f7cdb94ea None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.946849 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c3150-6e9e-429c-b637-ad9f7cdb94ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.947613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: WARNING py.warnings [None req-0593bc69-30fa-4a96-b3d7-13a1442da6aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:55:31.947613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: warnings.warn(msg) Sep 14 19:55:31.947613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: Sep 14 19:55:31.949041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593bc69-30fa-4a96-b3d7-13a1442da6aa admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:31.965964 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-168c3150-6e9e-429c-b637-ad9f7cdb94ea None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:31.967240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-168c3150-6e9e-429c-b637-ad9f7cdb94ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:31.976076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 792/1558] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:31.977219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8164508-309e-4868-ab7c-09d963fb84da None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:31.977503 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8164508-309e-4868-ab7c-09d963fb84da None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:31.977822 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8164508-309e-4868-ab7c-09d963fb84da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:31.997349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b8164508-309e-4868-ab7c-09d963fb84da None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:31.998487 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b8164508-309e-4868-ab7c-09d963fb84da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:32.006322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 793/1559] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:32.007530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdd23c-3a0e-4168-80d6-887ee8293058 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.007811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdd23c-3a0e-4168-80d6-887ee8293058 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.008079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfdd23c-3a0e-4168-80d6-887ee8293058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.026648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0cfdd23c-3a0e-4168-80d6-887ee8293058 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:32.027809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0cfdd23c-3a0e-4168-80d6-887ee8293058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:32.036114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 794/1560] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:32.037225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b051434-52c7-440b-b740-3d27ca5d1c88 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.037578 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b051434-52c7-440b-b740-3d27ca5d1c88 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.037871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b051434-52c7-440b-b740-3d27ca5d1c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.044940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:32.056665 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 767/1561] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:55:31 2022] POST /identity/v3/users => generated 296 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:32.059871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593bc69-30fa-4a96-b3d7-13a1442da6aa admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.060000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-9b051434-52c7-440b-b740-3d27ca5d1c88 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:32.060000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9b051434-52c7-440b-b740-3d27ca5d1c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:32.067445 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593bc69-30fa-4a96-b3d7-13a1442da6aa admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.068029 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 795/1562] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:32.068813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b051434-52c7-440b-b740-3d27ca5d1c88 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.075580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02443106-5351-49c1-b416-e3a97151fa99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.076175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02443106-5351-49c1-b416-e3a97151fa99 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.076451 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02443106-5351-49c1-b416-e3a97151fa99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.076720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02443106-5351-49c1-b416-e3a97151fa99 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.077354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02443106-5351-49c1-b416-e3a97151fa99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:55:32.077677 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02443106-5351-49c1-b416-e3a97151fa99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.078583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b051434-52c7-440b-b740-3d27ca5d1c88 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.079310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02443106-5351-49c1-b416-e3a97151fa99 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.083426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f402fa1-4ad1-435a-8bb0-a179831d3c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.088459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f402fa1-4ad1-435a-8bb0-a179831d3c73 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.088459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f402fa1-4ad1-435a-8bb0-a179831d3c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.088459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f402fa1-4ad1-435a-8bb0-a179831d3c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.088459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f402fa1-4ad1-435a-8bb0-a179831d3c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.088827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:32.088933 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f402fa1-4ad1-435a-8bb0-a179831d3c73 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.092936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 768/1563] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:55:32.098240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 796/1564] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:32.103771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02443106-5351-49c1-b416-e3a97151fa99 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.104273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f402fa1-4ad1-435a-8bb0-a179831d3c73 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.113499 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02443106-5351-49c1-b416-e3a97151fa99 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.117361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f402fa1-4ad1-435a-8bb0-a179831d3c73 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.118862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1904d7e-e3a1-4b06-8303-a9e03fd9ed3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.119665 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1904d7e-e3a1-4b06-8303-a9e03fd9ed3c admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.119955 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1904d7e-e3a1-4b06-8303-a9e03fd9ed3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.120148 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319692cb-1dd9-4699-bd6a-a1bd1c2f9023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.120268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1904d7e-e3a1-4b06-8303-a9e03fd9ed3c admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.120780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319692cb-1dd9-4699-bd6a-a1bd1c2f9023 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.120895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1904d7e-e3a1-4b06-8303-a9e03fd9ed3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:55:32.121048 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319692cb-1dd9-4699-bd6a-a1bd1c2f9023 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.121269 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1904d7e-e3a1-4b06-8303-a9e03fd9ed3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.121391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319692cb-1dd9-4699-bd6a-a1bd1c2f9023 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.121997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319692cb-1dd9-4699-bd6a-a1bd1c2f9023 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:55:32.122296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319692cb-1dd9-4699-bd6a-a1bd1c2f9023 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.123088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1904d7e-e3a1-4b06-8303-a9e03fd9ed3c admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.123928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319692cb-1dd9-4699-bd6a-a1bd1c2f9023 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.135109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 769/1565] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:55:32.142514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 797/1566] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:55:32.142684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1904d7e-e3a1-4b06-8303-a9e03fd9ed3c admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.142773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319692cb-1dd9-4699-bd6a-a1bd1c2f9023 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.146064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1904d7e-e3a1-4b06-8303-a9e03fd9ed3c admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.149743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319692cb-1dd9-4699-bd6a-a1bd1c2f9023 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.150684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31900e0-637f-42a6-85f3-a8be7cb8b0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.151276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31900e0-637f-42a6-85f3-a8be7cb8b0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.151557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31900e0-637f-42a6-85f3-a8be7cb8b0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.151830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31900e0-637f-42a6-85f3-a8be7cb8b0a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.152384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23efe19-ad91-4931-944e-f19260fa60a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.152520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31900e0-637f-42a6-85f3-a8be7cb8b0a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:55:32.152747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31900e0-637f-42a6-85f3-a8be7cb8b0a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.152986 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23efe19-ad91-4931-944e-f19260fa60a1 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.153419 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23efe19-ad91-4931-944e-f19260fa60a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.153699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23efe19-ad91-4931-944e-f19260fa60a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.154367 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23efe19-ad91-4931-944e-f19260fa60a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:55:32.154490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31900e0-637f-42a6-85f3-a8be7cb8b0a3 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.154669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23efe19-ad91-4931-944e-f19260fa60a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.156774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23efe19-ad91-4931-944e-f19260fa60a1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.174246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 770/1567] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:55:32.174246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31900e0-637f-42a6-85f3-a8be7cb8b0a3 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.180391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 798/1568] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:55:32.180391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23efe19-ad91-4931-944e-f19260fa60a1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.180391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23efe19-ad91-4931-944e-f19260fa60a1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.184807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2019b-800b-4a01-b46e-80aa2688fa03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.185403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2019b-800b-4a01-b46e-80aa2688fa03 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.185684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2019b-800b-4a01-b46e-80aa2688fa03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.185990 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2019b-800b-4a01-b46e-80aa2688fa03 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.186567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31900e0-637f-42a6-85f3-a8be7cb8b0a3 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.186689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2019b-800b-4a01-b46e-80aa2688fa03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:55:32.187506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2019b-800b-4a01-b46e-80aa2688fa03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.189096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b2019b-800b-4a01-b46e-80aa2688fa03 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.192050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453f61c-f1f5-429c-a451-77700c407253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.192641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453f61c-f1f5-429c-a451-77700c407253 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.192920 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453f61c-f1f5-429c-a451-77700c407253 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.193196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b453f61c-f1f5-429c-a451-77700c407253 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.193844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453f61c-f1f5-429c-a451-77700c407253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.195420 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b453f61c-f1f5-429c-a451-77700c407253 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.202252 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 799/1569] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:55:32.203680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2019b-800b-4a01-b46e-80aa2688fa03 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.212881 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 771/1570] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:32.213971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2019b-800b-4a01-b46e-80aa2688fa03 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.214760 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453f61c-f1f5-429c-a451-77700c407253 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.218741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3dc09-08a4-4d9e-b5ff-2322d6490ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.219332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3dc09-08a4-4d9e-b5ff-2322d6490ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.219608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3dc09-08a4-4d9e-b5ff-2322d6490ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.219877 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3dc09-08a4-4d9e-b5ff-2322d6490ba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.220543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3dc09-08a4-4d9e-b5ff-2322d6490ba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.222487 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3dc09-08a4-4d9e-b5ff-2322d6490ba9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.224852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b453f61c-f1f5-429c-a451-77700c407253 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.229480 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243bc10-9ec8-4e59-aec2-e5b23f183363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.230233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243bc10-9ec8-4e59-aec2-e5b23f183363 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.237709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243bc10-9ec8-4e59-aec2-e5b23f183363 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.237709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6243bc10-9ec8-4e59-aec2-e5b23f183363 admin admin] PATH_INFO: `/v3/projects/b04e9cb110f049d69a2d2747c74c8b51/users/96428b8e76314536a9e7033a841454e2/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.246199 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243bc10-9ec8-4e59-aec2-e5b23f183363 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04e9cb110f049d69a2d2747c74c8b51, user_id=96428b8e76314536a9e7033a841454e2, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.248898 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6243bc10-9ec8-4e59-aec2-e5b23f183363 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.347777 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 772/1571] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:32 2022] PUT /identity/v3/projects/b04e9cb110f049d69a2d2747c74c8b51/users/96428b8e76314536a9e7033a841454e2/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:32.348591 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243bc10-9ec8-4e59-aec2-e5b23f183363 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.349270 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 800/1572] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:55:32 2022] POST /identity/v3/projects => generated 387 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:32.349997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3dc09-08a4-4d9e-b5ff-2322d6490ba9 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.412477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6243bc10-9ec8-4e59-aec2-e5b23f183363 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.413581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3dc09-08a4-4d9e-b5ff-2322d6490ba9 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.417164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46e48c-314b-4141-a2f3-e30072ae8acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.417764 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46e48c-314b-4141-a2f3-e30072ae8acf admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.418068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46e48c-314b-4141-a2f3-e30072ae8acf admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.418170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba879ebc-5a01-4ee9-a9a2-55f7858db2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.418337 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d46e48c-314b-4141-a2f3-e30072ae8acf admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.418760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba879ebc-5a01-4ee9-a9a2-55f7858db2b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.418960 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46e48c-314b-4141-a2f3-e30072ae8acf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.419066 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba879ebc-5a01-4ee9-a9a2-55f7858db2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.419283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba879ebc-5a01-4ee9-a9a2-55f7858db2b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.419891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba879ebc-5a01-4ee9-a9a2-55f7858db2b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.420871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d46e48c-314b-4141-a2f3-e30072ae8acf admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.422296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba879ebc-5a01-4ee9-a9a2-55f7858db2b0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.513137 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 801/1573] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:55:32 2022] POST /identity/v3/projects => generated 393 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:32.513895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba879ebc-5a01-4ee9-a9a2-55f7858db2b0 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.541641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 773/1574] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:55:32 2022] POST /identity/v3/projects => generated 383 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:32.542476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46e48c-314b-4141-a2f3-e30072ae8acf admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.579928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba879ebc-5a01-4ee9-a9a2-55f7858db2b0 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.592240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:32.604471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d46e48c-314b-4141-a2f3-e30072ae8acf admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.609553 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdf724-a5ed-4855-9516-228ed9aa893c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.610181 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdf724-a5ed-4855-9516-228ed9aa893c admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.610454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdf724-a5ed-4855-9516-228ed9aa893c admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.610716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdf724-a5ed-4855-9516-228ed9aa893c admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.611329 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fdf724-a5ed-4855-9516-228ed9aa893c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.613232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fdf724-a5ed-4855-9516-228ed9aa893c admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.637514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b2761-49b9-44d6-9985-f88093a4e6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.639368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b2761-49b9-44d6-9985-f88093a4e6cf admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.639368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b2761-49b9-44d6-9985-f88093a4e6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.639368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b2761-49b9-44d6-9985-f88093a4e6cf admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.639368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b2761-49b9-44d6-9985-f88093a4e6cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.641345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b2761-49b9-44d6-9985-f88093a4e6cf admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.691981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 774/1575] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:55:32 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:32.693040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdf724-a5ed-4855-9516-228ed9aa893c admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.701619 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:32.720885 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 802/1576] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:55:32 2022] POST /identity/v3/projects => generated 367 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:32.725550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b2761-49b9-44d6-9985-f88093a4e6cf admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.756708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdf724-a5ed-4855-9516-228ed9aa893c admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.782725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b2761-49b9-44d6-9985-f88093a4e6cf admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.787368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92416303-f34d-48d4-8606-02bed359a034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.787987 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92416303-f34d-48d4-8606-02bed359a034 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.788262 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92416303-f34d-48d4-8606-02bed359a034 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.788566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92416303-f34d-48d4-8606-02bed359a034 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.789257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92416303-f34d-48d4-8606-02bed359a034 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.790917 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92416303-f34d-48d4-8606-02bed359a034 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.801430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 803/1577] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:32.805897 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3800e9-f6de-4828-b7ce-8fd207f11dcb None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.806185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3800e9-f6de-4828-b7ce-8fd207f11dcb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.808263 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3800e9-f6de-4828-b7ce-8fd207f11dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.814016 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e837d1-aabb-44c4-ba54-1b75c94a8de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.814858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e837d1-aabb-44c4-ba54-1b75c94a8de6 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.815133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e837d1-aabb-44c4-ba54-1b75c94a8de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.815397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e837d1-aabb-44c4-ba54-1b75c94a8de6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.816015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e837d1-aabb-44c4-ba54-1b75c94a8de6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.817924 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e837d1-aabb-44c4-ba54-1b75c94a8de6 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.825487 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1d3800e9-f6de-4828-b7ce-8fd207f11dcb None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:32.826779 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1d3800e9-f6de-4828-b7ce-8fd207f11dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:32.834660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 804/1578] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:32.835412 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3800e9-f6de-4828-b7ce-8fd207f11dcb None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.845488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3800e9-f6de-4828-b7ce-8fd207f11dcb None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:32.850213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf19d18-5134-4a47-a32a-4fd6586e212d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:32.850800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf19d18-5134-4a47-a32a-4fd6586e212d admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:32.851079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf19d18-5134-4a47-a32a-4fd6586e212d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:32.851353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf19d18-5134-4a47-a32a-4fd6586e212d admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:32.851970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf19d18-5134-4a47-a32a-4fd6586e212d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:32.853906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf19d18-5134-4a47-a32a-4fd6586e212d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:32.943650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 775/1579] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:55:32 2022] POST /identity/v3/projects => generated 371 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:32.944620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e837d1-aabb-44c4-ba54-1b75c94a8de6 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.981091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 805/1580] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:55:32 2022] POST /identity/v3/users => generated 294 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:32.981962 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf19d18-5134-4a47-a32a-4fd6586e212d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:32.992038 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf19d18-5134-4a47-a32a-4fd6586e212d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.007729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e837d1-aabb-44c4-ba54-1b75c94a8de6 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.012340 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cf16c-3a80-4cb5-8e9a-b8499e3a57a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.012881 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cf16c-3a80-4cb5-8e9a-b8499e3a57a6 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.013157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cf16c-3a80-4cb5-8e9a-b8499e3a57a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.013426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04cf16c-3a80-4cb5-8e9a-b8499e3a57a6 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.014072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04cf16c-3a80-4cb5-8e9a-b8499e3a57a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.016497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04cf16c-3a80-4cb5-8e9a-b8499e3a57a6 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.048148 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e847f-1ecb-485b-82d8-8289c8102efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.048753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e847f-1ecb-485b-82d8-8289c8102efd admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.049025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e847f-1ecb-485b-82d8-8289c8102efd admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.049293 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e847f-1ecb-485b-82d8-8289c8102efd admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.049936 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e847f-1ecb-485b-82d8-8289c8102efd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.051865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e847f-1ecb-485b-82d8-8289c8102efd admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.096231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 776/1581] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:55:32 2022] POST /identity/v3/users => generated 297 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:33.097036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cf16c-3a80-4cb5-8e9a-b8499e3a57a6 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.106415 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04cf16c-3a80-4cb5-8e9a-b8499e3a57a6 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.111042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd9394-a9f8-4e83-8045-f43121b3c06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.111613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd9394-a9f8-4e83-8045-f43121b3c06f admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.111891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd9394-a9f8-4e83-8045-f43121b3c06f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.112163 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd9394-a9f8-4e83-8045-f43121b3c06f admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.112760 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd9394-a9f8-4e83-8045-f43121b3c06f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.114778 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccd9394-a9f8-4e83-8045-f43121b3c06f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.163817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 806/1582] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:55:32 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:33.164657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e847f-1ecb-485b-82d8-8289c8102efd admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.169466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:33.174516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e847f-1ecb-485b-82d8-8289c8102efd admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.179138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd77e3f-6230-42b0-b4eb-3495ab1153fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.179690 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd77e3f-6230-42b0-b4eb-3495ab1153fb admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.179967 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd77e3f-6230-42b0-b4eb-3495ab1153fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.180240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd77e3f-6230-42b0-b4eb-3495ab1153fb admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.180846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd77e3f-6230-42b0-b4eb-3495ab1153fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.182795 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd77e3f-6230-42b0-b4eb-3495ab1153fb admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.215128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 777/1583] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:55:33 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:33.215938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd9394-a9f8-4e83-8045-f43121b3c06f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.225971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccd9394-a9f8-4e83-8045-f43121b3c06f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.230541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd7a24-fa82-48ac-86dd-e03aeeb1738d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.231158 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd7a24-fa82-48ac-86dd-e03aeeb1738d admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.231436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd7a24-fa82-48ac-86dd-e03aeeb1738d admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.231736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd7a24-fa82-48ac-86dd-e03aeeb1738d admin admin] PATH_INFO: `/v3/projects/e1dc64fa9e8e422fab6b3a66a29278ff/users/766dd5a650fc4855980250969854f08d/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.240183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:33.246425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd7a24-fa82-48ac-86dd-e03aeeb1738d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1dc64fa9e8e422fab6b3a66a29278ff, user_id=766dd5a650fc4855980250969854f08d, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.249053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd7a24-fa82-48ac-86dd-e03aeeb1738d admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.302670 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 807/1584] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:55:33 2022] POST /identity/v3/users => generated 284 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:33.303549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd77e3f-6230-42b0-b4eb-3495ab1153fb admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.342171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 778/1585] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:33 2022] PUT /identity/v3/projects/e1dc64fa9e8e422fab6b3a66a29278ff/users/766dd5a650fc4855980250969854f08d/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:33.342973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd7a24-fa82-48ac-86dd-e03aeeb1738d admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.360200 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd77e3f-6230-42b0-b4eb-3495ab1153fb admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.404552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd7a24-fa82-48ac-86dd-e03aeeb1738d admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.409195 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d29d6-cf84-47c2-9a4e-3995ee9a3b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.409848 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d29d6-cf84-47c2-9a4e-3995ee9a3b2a admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.410037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e1d23-01c1-4c0d-9148-32c535be47b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.410157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d29d6-cf84-47c2-9a4e-3995ee9a3b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.410379 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d29d6-cf84-47c2-9a4e-3995ee9a3b2a admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.410646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e1d23-01c1-4c0d-9148-32c535be47b9 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.410919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e1d23-01c1-4c0d-9148-32c535be47b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.411106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d29d6-cf84-47c2-9a4e-3995ee9a3b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.411211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e1d23-01c1-4c0d-9148-32c535be47b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.413017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d29d6-cf84-47c2-9a4e-3995ee9a3b2a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.420873 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e1d23-01c1-4c0d-9148-32c535be47b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.422361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e1d23-01c1-4c0d-9148-32c535be47b9 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.431545 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 808/1586] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:33.438092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e1d23-01c1-4c0d-9148-32c535be47b9 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.442257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e1d23-01c1-4c0d-9148-32c535be47b9 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.446938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088c38c-79cc-40cd-9669-b4e662c80663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.447518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088c38c-79cc-40cd-9669-b4e662c80663 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.447925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088c38c-79cc-40cd-9669-b4e662c80663 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.448197 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088c38c-79cc-40cd-9669-b4e662c80663 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.448808 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088c38c-79cc-40cd-9669-b4e662c80663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.450539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088c38c-79cc-40cd-9669-b4e662c80663 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.461143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 809/1587] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:33.462036 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088c38c-79cc-40cd-9669-b4e662c80663 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.471948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088c38c-79cc-40cd-9669-b4e662c80663 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.476718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab1c37-0123-4a2c-be0e-8c972de67a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.477310 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1c37-0123-4a2c-be0e-8c972de67a95 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.477590 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1c37-0123-4a2c-be0e-8c972de67a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.478031 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab1c37-0123-4a2c-be0e-8c972de67a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.478568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab1c37-0123-4a2c-be0e-8c972de67a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.480210 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab1c37-0123-4a2c-be0e-8c972de67a95 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.490048 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 810/1588] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:33.490894 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab1c37-0123-4a2c-be0e-8c972de67a95 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.501299 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab1c37-0123-4a2c-be0e-8c972de67a95 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.506166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66339425-ff26-4af8-be91-721c8f570e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.506869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66339425-ff26-4af8-be91-721c8f570e03 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.507203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66339425-ff26-4af8-be91-721c8f570e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.507571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66339425-ff26-4af8-be91-721c8f570e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.508295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66339425-ff26-4af8-be91-721c8f570e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.508418 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 779/1589] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:55:33 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:33.509160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d29d6-cf84-47c2-9a4e-3995ee9a3b2a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.510170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66339425-ff26-4af8-be91-721c8f570e03 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.520267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d29d6-cf84-47c2-9a4e-3995ee9a3b2a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.521832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 811/1590] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:33.522633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66339425-ff26-4af8-be91-721c8f570e03 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.525947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69198eb7-ad3a-450b-bfb8-ff634f60583c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.526762 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69198eb7-ad3a-450b-bfb8-ff634f60583c admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.527228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69198eb7-ad3a-450b-bfb8-ff634f60583c admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.527609 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69198eb7-ad3a-450b-bfb8-ff634f60583c admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.528406 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69198eb7-ad3a-450b-bfb8-ff634f60583c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.530395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69198eb7-ad3a-450b-bfb8-ff634f60583c admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.532183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66339425-ff26-4af8-be91-721c8f570e03 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.537655 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0e525-f5b3-4217-9922-82d4b6d18abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.538279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e525-f5b3-4217-9922-82d4b6d18abf admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.538553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e525-f5b3-4217-9922-82d4b6d18abf admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.538832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e525-f5b3-4217-9922-82d4b6d18abf admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.539443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0e525-f5b3-4217-9922-82d4b6d18abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.541024 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0e525-f5b3-4217-9922-82d4b6d18abf admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.543186 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 780/1591] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:33.544911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf428e-3929-4844-a2f7-cc68ae9bc81d None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.545267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf428e-3929-4844-a2f7-cc68ae9bc81d None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.545955 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf428e-3929-4844-a2f7-cc68ae9bc81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.553988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 812/1592] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:33.554780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0e525-f5b3-4217-9922-82d4b6d18abf admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.565704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0e525-f5b3-4217-9922-82d4b6d18abf admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.570010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-93bf428e-3929-4844-a2f7-cc68ae9bc81d None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:33.571249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-93bf428e-3929-4844-a2f7-cc68ae9bc81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:33.571915 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba783d2-86ba-4764-bab9-750423110470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.572532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba783d2-86ba-4764-bab9-750423110470 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.572819 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba783d2-86ba-4764-bab9-750423110470 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.573144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba783d2-86ba-4764-bab9-750423110470 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.573741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba783d2-86ba-4764-bab9-750423110470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.575321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba783d2-86ba-4764-bab9-750423110470 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.579962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 781/1593] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:33.580753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf428e-3929-4844-a2f7-cc68ae9bc81d None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.591848 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 813/1594] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:33.591848 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba783d2-86ba-4764-bab9-750423110470 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.597384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf428e-3929-4844-a2f7-cc68ae9bc81d None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.602400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba783d2-86ba-4764-bab9-750423110470 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.604387 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2eabf-379d-4344-8d32-af3c2d0bf915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.605129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2eabf-379d-4344-8d32-af3c2d0bf915 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.605331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2eabf-379d-4344-8d32-af3c2d0bf915 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.605602 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2eabf-379d-4344-8d32-af3c2d0bf915 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.606764 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2eabf-379d-4344-8d32-af3c2d0bf915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.607914 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c2eabf-379d-4344-8d32-af3c2d0bf915 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.608359 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1003577-0434-4b75-8a15-6312758e9747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.609069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1003577-0434-4b75-8a15-6312758e9747 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.609361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1003577-0434-4b75-8a15-6312758e9747 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.609633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1003577-0434-4b75-8a15-6312758e9747 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.610407 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1003577-0434-4b75-8a15-6312758e9747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.612057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1003577-0434-4b75-8a15-6312758e9747 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.621050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 782/1595] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:33.622367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2eabf-379d-4344-8d32-af3c2d0bf915 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.625995 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 814/1596] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:33.627071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1003577-0434-4b75-8a15-6312758e9747 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.635766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2eabf-379d-4344-8d32-af3c2d0bf915 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.641534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb65be-0abe-41fc-b88f-ec69aa6f4bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.642210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb65be-0abe-41fc-b88f-ec69aa6f4bac admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.642487 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb65be-0abe-41fc-b88f-ec69aa6f4bac admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.642758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb65be-0abe-41fc-b88f-ec69aa6f4bac admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.643703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb65be-0abe-41fc-b88f-ec69aa6f4bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.644309 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1003577-0434-4b75-8a15-6312758e9747 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.645369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb65be-0abe-41fc-b88f-ec69aa6f4bac admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.649876 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb53ae-64fe-47b2-9b7b-9f4e7fbed385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.650540 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb53ae-64fe-47b2-9b7b-9f4e7fbed385 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.654535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb53ae-64fe-47b2-9b7b-9f4e7fbed385 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.654535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb53ae-64fe-47b2-9b7b-9f4e7fbed385 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.654535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb53ae-64fe-47b2-9b7b-9f4e7fbed385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.654535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb53ae-64fe-47b2-9b7b-9f4e7fbed385 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.657836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 783/1597] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:33.657836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb65be-0abe-41fc-b88f-ec69aa6f4bac admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.664133 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 815/1598] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:33.664783 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb53ae-64fe-47b2-9b7b-9f4e7fbed385 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.666647 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-befb53ae-64fe-47b2-9b7b-9f4e7fbed385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:33.669577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb65be-0abe-41fc-b88f-ec69aa6f4bac admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.674518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7690dfd-8a34-46f1-a39c-ef03fddc8e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.675123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7690dfd-8a34-46f1-a39c-ef03fddc8e84 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.675404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7690dfd-8a34-46f1-a39c-ef03fddc8e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.675676 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7690dfd-8a34-46f1-a39c-ef03fddc8e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.676314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7690dfd-8a34-46f1-a39c-ef03fddc8e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.677927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7690dfd-8a34-46f1-a39c-ef03fddc8e84 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.678408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb53ae-64fe-47b2-9b7b-9f4e7fbed385 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.682092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f834d7-0f86-490e-b086-b0d1a1f77edc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.683183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f834d7-0f86-490e-b086-b0d1a1f77edc service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.683183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f834d7-0f86-490e-b086-b0d1a1f77edc service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.683183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f834d7-0f86-490e-b086-b0d1a1f77edc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.689450 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 784/1599] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:33.690579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7690dfd-8a34-46f1-a39c-ef03fddc8e84 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.696258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f834d7-0f86-490e-b086-b0d1a1f77edc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.697692 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f834d7-0f86-490e-b086-b0d1a1f77edc service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.700106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7690dfd-8a34-46f1-a39c-ef03fddc8e84 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.705646 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa2b70-7355-4f13-82eb-6280429d90a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.706539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa2b70-7355-4f13-82eb-6280429d90a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.706850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa2b70-7355-4f13-82eb-6280429d90a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.707132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa2b70-7355-4f13-82eb-6280429d90a8 admin admin] PATH_INFO: `/v3/projects/bdb5ca35665545119e4ca1f379c37a73/users/2bf1c7a6cc2e4b1294e7b162a8aba738/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.708200 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 816/1600] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:33.709478 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f834d7-0f86-490e-b086-b0d1a1f77edc service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.718881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f834d7-0f86-490e-b086-b0d1a1f77edc service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.723254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa2b70-7355-4f13-82eb-6280429d90a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb5ca35665545119e4ca1f379c37a73, user_id=2bf1c7a6cc2e4b1294e7b162a8aba738, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.724039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cbdaf-1d4c-4bd1-b225-501d5d8bc066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.724785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cbdaf-1d4c-4bd1-b225-501d5d8bc066 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.725255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cbdaf-1d4c-4bd1-b225-501d5d8bc066 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.725599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890cbdaf-1d4c-4bd1-b225-501d5d8bc066 admin admin] PATH_INFO: `/v3/projects/3240927f0b59480d9e218342273b33c9/users/ad18d3c6633a47538f29dfd0d2e9db16/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.726182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa2b70-7355-4f13-82eb-6280429d90a8 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.739568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:33.743845 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890cbdaf-1d4c-4bd1-b225-501d5d8bc066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3240927f0b59480d9e218342273b33c9, user_id=ad18d3c6633a47538f29dfd0d2e9db16, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.744927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-890cbdaf-1d4c-4bd1-b225-501d5d8bc066 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:55:33.744927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Sep 14 19:55:33.744927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: Sep 14 19:55:33.747068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890cbdaf-1d4c-4bd1-b225-501d5d8bc066 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.811784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 785/1601] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:33 2022] PUT /identity/v3/projects/bdb5ca35665545119e4ca1f379c37a73/users/2bf1c7a6cc2e4b1294e7b162a8aba738/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:33.816836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa2b70-7355-4f13-82eb-6280429d90a8 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.867454 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 817/1602] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:33 2022] PUT /identity/v3/projects/3240927f0b59480d9e218342273b33c9/users/ad18d3c6633a47538f29dfd0d2e9db16/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:33.869528 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cbdaf-1d4c-4bd1-b225-501d5d8bc066 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.906904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa2b70-7355-4f13-82eb-6280429d90a8 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.912341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:33.947472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890cbdaf-1d4c-4bd1-b225-501d5d8bc066 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.952107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f8654-327d-4304-aa78-c650602ac1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.952794 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f8654-327d-4304-aa78-c650602ac1e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.953075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f8654-327d-4304-aa78-c650602ac1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.953348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f8654-327d-4304-aa78-c650602ac1e0 admin admin] PATH_INFO: `/v3/projects/98ab5a8a92a04992a4ad32e96f32306c/users/01d6926769014a88bbfd4be361b2a307/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.964728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635018a7-5970-4c29-bbc8-0bef9d6f4a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.965426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635018a7-5970-4c29-bbc8-0bef9d6f4a9b admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.965703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635018a7-5970-4c29-bbc8-0bef9d6f4a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.966009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635018a7-5970-4c29-bbc8-0bef9d6f4a9b admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:33.966641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635018a7-5970-4c29-bbc8-0bef9d6f4a9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.968059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f8654-327d-4304-aa78-c650602ac1e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98ab5a8a92a04992a4ad32e96f32306c, user_id=01d6926769014a88bbfd4be361b2a307, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:33.968256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635018a7-5970-4c29-bbc8-0bef9d6f4a9b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.970814 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f8654-327d-4304-aa78-c650602ac1e0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:33.977906 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 786/1603] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:33.978710 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635018a7-5970-4c29-bbc8-0bef9d6f4a9b admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:33.989003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635018a7-5970-4c29-bbc8-0bef9d6f4a9b admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:33.993716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d161a8-e801-4012-ae1f-a4ed999b5adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:33.994392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d161a8-e801-4012-ae1f-a4ed999b5adb admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:33.994671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d161a8-e801-4012-ae1f-a4ed999b5adb admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:33.994940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d161a8-e801-4012-ae1f-a4ed999b5adb admin admin] PATH_INFO: `/v3/projects/e52433815b1440de8bfe05dc7d14b7f5/users/268dbf54f29c40d9a4daa772c44735d4/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.014484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d161a8-e801-4012-ae1f-a4ed999b5adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e52433815b1440de8bfe05dc7d14b7f5, user_id=268dbf54f29c40d9a4daa772c44735d4, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:34.021450 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d161a8-e801-4012-ae1f-a4ed999b5adb admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:34.058288 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 818/1604] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:33 2022] PUT /identity/v3/projects/98ab5a8a92a04992a4ad32e96f32306c/users/01d6926769014a88bbfd4be361b2a307/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:34.059170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f8654-327d-4304-aa78-c650602ac1e0 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:34.086832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 787/1605] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:33 2022] PUT /identity/v3/projects/e52433815b1440de8bfe05dc7d14b7f5/users/268dbf54f29c40d9a4daa772c44735d4/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:34.088415 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e8643-6bd5-455b-867c-ca97ba8161cd None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:34.088847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e8643-6bd5-455b-867c-ca97ba8161cd None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:34.089129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e8643-6bd5-455b-867c-ca97ba8161cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.125140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-529e8643-6bd5-455b-867c-ca97ba8161cd None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:34.126479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-529e8643-6bd5-455b-867c-ca97ba8161cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:34.172657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f8654-327d-4304-aa78-c650602ac1e0 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:34.206756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 788/1606] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:34.208006 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714132ed-ffcc-41bf-97af-ecdf92ceda13 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:34.208296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714132ed-ffcc-41bf-97af-ecdf92ceda13 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:34.208568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714132ed-ffcc-41bf-97af-ecdf92ceda13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.237787 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-714132ed-ffcc-41bf-97af-ecdf92ceda13 None None] MFA Rules not processed for user `ad18d3c6633a47538f29dfd0d2e9db16`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:34.239124 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-714132ed-ffcc-41bf-97af-ecdf92ceda13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:34.240230 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494905c-9183-48fd-9dca-b91aa4bbbcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:34.240902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494905c-9183-48fd-9dca-b91aa4bbbcac admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:34.241184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494905c-9183-48fd-9dca-b91aa4bbbcac admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:34.241456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494905c-9183-48fd-9dca-b91aa4bbbcac admin admin] PATH_INFO: `/v3/projects/28a7633a75a745049c541513cd4b1120/users/0e73f6986ae247369ab904edea2b51a0/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.268562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494905c-9183-48fd-9dca-b91aa4bbbcac admin admin] RBAC: Authorizing `identity:create_grant(project_id=28a7633a75a745049c541513cd4b1120, user_id=0e73f6986ae247369ab904edea2b51a0, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:34.271609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9494905c-9183-48fd-9dca-b91aa4bbbcac admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:34.327548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 789/1607] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:34.329291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714132ed-ffcc-41bf-97af-ecdf92ceda13 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:34.336643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 819/1608] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:34 2022] PUT /identity/v3/projects/28a7633a75a745049c541513cd4b1120/users/0e73f6986ae247369ab904edea2b51a0/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:34.338618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7190dec-e337-4e9b-9f41-c850695cb17f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:34.339410 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7190dec-e337-4e9b-9f41-c850695cb17f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:34.340151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7190dec-e337-4e9b-9f41-c850695cb17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.349069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714132ed-ffcc-41bf-97af-ecdf92ceda13 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:34.352808 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:34.363702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:34.385375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e7190dec-e337-4e9b-9f41-c850695cb17f None None] MFA Rules not processed for user `01d6926769014a88bbfd4be361b2a307`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:34.386998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e7190dec-e337-4e9b-9f41-c850695cb17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:34.451522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924f28a-e58f-4a39-bdef-a45975b7a336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:34.452369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924f28a-e58f-4a39-bdef-a45975b7a336 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:34.453050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924f28a-e58f-4a39-bdef-a45975b7a336 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:34.453505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924f28a-e58f-4a39-bdef-a45975b7a336 admin admin] PATH_INFO: `/v3/projects/247adb0616cb456bb31122a8921cdf73/users/07c601b050c4428381ccf74d805f7bf6/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.472805 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924f28a-e58f-4a39-bdef-a45975b7a336 admin admin] RBAC: Authorizing `identity:create_grant(project_id=247adb0616cb456bb31122a8921cdf73, user_id=07c601b050c4428381ccf74d805f7bf6, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:34.478584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924f28a-e58f-4a39-bdef-a45975b7a336 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:34.480090 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 820/1609] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:34.481637 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1daf3f-93ad-40fc-8e33-bb0946b35e74 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:34.482237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1daf3f-93ad-40fc-8e33-bb0946b35e74 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:34.483020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1daf3f-93ad-40fc-8e33-bb0946b35e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.506059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3f1daf3f-93ad-40fc-8e33-bb0946b35e74 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:34.508751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3f1daf3f-93ad-40fc-8e33-bb0946b35e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:34.518733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 821/1610] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:34.519643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1daf3f-93ad-40fc-8e33-bb0946b35e74 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:34.532489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 790/1611] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:34 2022] PUT /identity/v3/projects/247adb0616cb456bb31122a8921cdf73/users/07c601b050c4428381ccf74d805f7bf6/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:34.537673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924f28a-e58f-4a39-bdef-a45975b7a336 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:34.601781 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1daf3f-93ad-40fc-8e33-bb0946b35e74 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:34.616104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924f28a-e58f-4a39-bdef-a45975b7a336 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:34.620868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be401561-1d7c-447f-a5e5-0fa54f2596cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:34.621479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be401561-1d7c-447f-a5e5-0fa54f2596cf admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:34.621760 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be401561-1d7c-447f-a5e5-0fa54f2596cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:34.622066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be401561-1d7c-447f-a5e5-0fa54f2596cf admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.622690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be401561-1d7c-447f-a5e5-0fa54f2596cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:34.625216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be401561-1d7c-447f-a5e5-0fa54f2596cf admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:34.653323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79b8e93-b816-4bfb-9301-cc9f8d888562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:34.654034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b8e93-b816-4bfb-9301-cc9f8d888562 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:34.654316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b8e93-b816-4bfb-9301-cc9f8d888562 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:34.654591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b8e93-b816-4bfb-9301-cc9f8d888562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.667776 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b8e93-b816-4bfb-9301-cc9f8d888562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:34.669674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79b8e93-b816-4bfb-9301-cc9f8d888562 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:34.680584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 822/1612] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:34.682143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7e9aa-0068-456d-a1ba-c08507ffc581 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:34.682143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7e9aa-0068-456d-a1ba-c08507ffc581 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:34.682364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb7e9aa-0068-456d-a1ba-c08507ffc581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.713693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-cfb7e9aa-0068-456d-a1ba-c08507ffc581 None None] MFA Rules not processed for user `0e73f6986ae247369ab904edea2b51a0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:34.714766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cfb7e9aa-0068-456d-a1ba-c08507ffc581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:34.759606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:34.762514 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 791/1613] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:55:34 2022] POST /identity/v3/projects => generated 393 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:34.766963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e249d86-5686-4047-81a8-eb0ff97b790e None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:34.767251 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e249d86-5686-4047-81a8-eb0ff97b790e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:34.767816 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e249d86-5686-4047-81a8-eb0ff97b790e None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.768930 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 792/1614] 10.222.0.42 () {54 vars in 962 bytes} [Wed Sep 14 19:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:55:34.769459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e249d86-5686-4047-81a8-eb0ff97b790e None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:34.791872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 823/1615] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:34.793110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b19d1-e11a-45c4-943a-b0f41300d150 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:34.793391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b19d1-e11a-45c4-943a-b0f41300d150 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:34.793665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b19d1-e11a-45c4-943a-b0f41300d150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.823494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-319b19d1-e11a-45c4-943a-b0f41300d150 None None] MFA Rules not processed for user `07c601b050c4428381ccf74d805f7bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:34.825229 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-319b19d1-e11a-45c4-943a-b0f41300d150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:34.830562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e249d86-5686-4047-81a8-eb0ff97b790e None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:34.834509 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee63c51-dbbd-4da0-8a19-6c34cd018ccc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:34.835068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee63c51-dbbd-4da0-8a19-6c34cd018ccc service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:34.835337 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee63c51-dbbd-4da0-8a19-6c34cd018ccc service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:34.835616 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee63c51-dbbd-4da0-8a19-6c34cd018ccc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.882897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:34.907275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 824/1616] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:34.907877 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b19d1-e11a-45c4-943a-b0f41300d150 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:34.909178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee63c51-dbbd-4da0-8a19-6c34cd018ccc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:34.910678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee63c51-dbbd-4da0-8a19-6c34cd018ccc service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:34.918910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b19d1-e11a-45c4-943a-b0f41300d150 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:34.920360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 793/1617] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:34.922255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a5a1f-c57e-4459-ad5c-35233810426e None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:34.922541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a5a1f-c57e-4459-ad5c-35233810426e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:34.922811 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a5a1f-c57e-4459-ad5c-35233810426e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.924716 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952ecba-7589-4e6b-8d56-7189aebcc90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:34.925325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952ecba-7589-4e6b-8d56-7189aebcc90f admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:34.925603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952ecba-7589-4e6b-8d56-7189aebcc90f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:34.925945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f952ecba-7589-4e6b-8d56-7189aebcc90f admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:34.926644 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952ecba-7589-4e6b-8d56-7189aebcc90f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:34.928550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f952ecba-7589-4e6b-8d56-7189aebcc90f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:34.961499 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-308a5a1f-c57e-4459-ad5c-35233810426e None None] MFA Rules not processed for user `2a4284b59f804dc59674d621d87634bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:34.962745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-308a5a1f-c57e-4459-ad5c-35233810426e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:35.046002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 794/1618] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Sep 14 19:55:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:35.049050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a5a1f-c57e-4459-ad5c-35233810426e None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:35.087600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 825/1619] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:55:34 2022] POST /identity/v3/users => generated 303 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:35.088700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952ecba-7589-4e6b-8d56-7189aebcc90f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:35.108007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f952ecba-7589-4e6b-8d56-7189aebcc90f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:35.117715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a04a5-4cb3-4adb-99c5-087ad74e2495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:35.119563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a04a5-4cb3-4adb-99c5-087ad74e2495 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:35.119846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a04a5-4cb3-4adb-99c5-087ad74e2495 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:35.120119 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a04a5-4cb3-4adb-99c5-087ad74e2495 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:35.120396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a5a1f-c57e-4459-ad5c-35233810426e None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:35.120911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a04a5-4cb3-4adb-99c5-087ad74e2495 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:35.126712 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a04a5-4cb3-4adb-99c5-087ad74e2495 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:35.127942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedffc2-ce06-48b3-b265-f37893580b4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:35.128648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedffc2-ce06-48b3-b265-f37893580b4d service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:35.129505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedffc2-ce06-48b3-b265-f37893580b4d service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:35.129885 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedffc2-ce06-48b3-b265-f37893580b4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:35.220486 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedffc2-ce06-48b3-b265-f37893580b4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:35.480469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedffc2-ce06-48b3-b265-f37893580b4d service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:35.480469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 795/1620] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:35 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:35.480469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedffc2-ce06-48b3-b265-f37893580b4d service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:35.480469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedffc2-ce06-48b3-b265-f37893580b4d service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:35.482285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe32fd-7142-4200-9498-ff840ecc504c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:35.482285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe32fd-7142-4200-9498-ff840ecc504c service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:35.482285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe32fd-7142-4200-9498-ff840ecc504c service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:35.482285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe32fd-7142-4200-9498-ff840ecc504c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:35.485386 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 826/1621] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:55:35 2022] POST /identity/v3/projects => generated 369 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:35.485386 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a04a5-4cb3-4adb-99c5-087ad74e2495 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:35.486109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe32fd-7142-4200-9498-ff840ecc504c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:35.486109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe32fd-7142-4200-9498-ff840ecc504c service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:35.486109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 796/1622] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:35.486109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe32fd-7142-4200-9498-ff840ecc504c service nova] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:35.488404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a04a5-4cb3-4adb-99c5-087ad74e2495 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:35.488404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545cad7-1078-4125-a004-7ac305b9204b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:35.488404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545cad7-1078-4125-a004-7ac305b9204b admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:35.488404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545cad7-1078-4125-a004-7ac305b9204b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:35.489005 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8545cad7-1078-4125-a004-7ac305b9204b admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:35.489005 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545cad7-1078-4125-a004-7ac305b9204b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:35.489137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe32fd-7142-4200-9498-ff840ecc504c service nova] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:35.489273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8545cad7-1078-4125-a004-7ac305b9204b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:35.489361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17218a7c-5bed-405b-8395-cd1ce76c6456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:35.489361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17218a7c-5bed-405b-8395-cd1ce76c6456 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:35.489361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17218a7c-5bed-405b-8395-cd1ce76c6456 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:35.489361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17218a7c-5bed-405b-8395-cd1ce76c6456 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:35.491915 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17218a7c-5bed-405b-8395-cd1ce76c6456 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:35.496587 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17218a7c-5bed-405b-8395-cd1ce76c6456 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:35.505883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 827/1623] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:35.508532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545cad7-1078-4125-a004-7ac305b9204b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:35.540013 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8545cad7-1078-4125-a004-7ac305b9204b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:35.560620 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde95531-d083-4c5e-8afd-d168ebf6c2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:35.561521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde95531-d083-4c5e-8afd-d168ebf6c2ae admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:35.561966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde95531-d083-4c5e-8afd-d168ebf6c2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:35.562988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde95531-d083-4c5e-8afd-d168ebf6c2ae admin admin] PATH_INFO: `/v3/projects/a4ed567f14174818a2e631956d791e87/users/56c24c9641a24308afd2de9cb337b415/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:35.594398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde95531-d083-4c5e-8afd-d168ebf6c2ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ed567f14174818a2e631956d791e87, user_id=56c24c9641a24308afd2de9cb337b415, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:35.599669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde95531-d083-4c5e-8afd-d168ebf6c2ae admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:35.607855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:35.642424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 797/1624] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:55:35 2022] POST /identity/v3/users => generated 291 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:35.645546 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17218a7c-5bed-405b-8395-cd1ce76c6456 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:35.671088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17218a7c-5bed-405b-8395-cd1ce76c6456 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:35.684051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3fe3a-e960-4488-b109-28c93a2c61aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:35.686034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3fe3a-e960-4488-b109-28c93a2c61aa admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:35.687980 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3fe3a-e960-4488-b109-28c93a2c61aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:35.688352 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3fe3a-e960-4488-b109-28c93a2c61aa admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:35.689718 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3fe3a-e960-4488-b109-28c93a2c61aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:35.694511 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3fe3a-e960-4488-b109-28c93a2c61aa admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:35.715079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 828/1625] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:35 2022] PUT /identity/v3/projects/a4ed567f14174818a2e631956d791e87/users/56c24c9641a24308afd2de9cb337b415/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:35.719395 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde95531-d083-4c5e-8afd-d168ebf6c2ae admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:35.721694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 798/1626] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:35.725952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3fe3a-e960-4488-b109-28c93a2c61aa admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:35.748689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3fe3a-e960-4488-b109-28c93a2c61aa admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:35.797313 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:35.810569 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde95531-d083-4c5e-8afd-d168ebf6c2ae admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:35.818835 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b352d7-36b0-4bd1-9cb9-1e476113a6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:35.819988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b352d7-36b0-4bd1-9cb9-1e476113a6af admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:35.820574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b352d7-36b0-4bd1-9cb9-1e476113a6af admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:35.821956 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b352d7-36b0-4bd1-9cb9-1e476113a6af admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:35.822871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b352d7-36b0-4bd1-9cb9-1e476113a6af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:35.825538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b352d7-36b0-4bd1-9cb9-1e476113a6af admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:35.842050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 829/1627] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:55:35.866809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd03532-27d7-465c-b479-7c5465d06113 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:35.876425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd03532-27d7-465c-b479-7c5465d06113 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:35.876745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd03532-27d7-465c-b479-7c5465d06113 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:35.880837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 830/1628] 10.222.0.42 () {54 vars in 912 bytes} [Wed Sep 14 19:55:35 2022] GET /identity => generated 270 bytes in 38 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:55:35.885508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd03532-27d7-465c-b479-7c5465d06113 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:35.902474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e578522-ba0c-4ad5-87b8-694fed68ee42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:35.903365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578522-ba0c-4ad5-87b8-694fed68ee42 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:35.904712 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578522-ba0c-4ad5-87b8-694fed68ee42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:35.905171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e578522-ba0c-4ad5-87b8-694fed68ee42 admin admin] PATH_INFO: `/v3/projects/034ba9252ef549ad84b9a4f24a7f115b/users/fc95d77e50fb48e09224a91e47f29feb/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:35.922219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd03532-27d7-465c-b479-7c5465d06113 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:35.931787 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac6f59a-8124-42d3-8425-3d10aa9a2ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:35.933215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6f59a-8124-42d3-8425-3d10aa9a2ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:35.933816 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6f59a-8124-42d3-8425-3d10aa9a2ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:35.934418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac6f59a-8124-42d3-8425-3d10aa9a2ef0 admin admin] PATH_INFO: `/v3/projects/a4ed567f14174818a2e631956d791e87/users/56c24c9641a24308afd2de9cb337b415/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:35.938453 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6f59a-8124-42d3-8425-3d10aa9a2ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4ed567f14174818a2e631956d791e87, user_id=56c24c9641a24308afd2de9cb337b415, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:35.942869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac6f59a-8124-42d3-8425-3d10aa9a2ef0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:35.943505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e578522-ba0c-4ad5-87b8-694fed68ee42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=034ba9252ef549ad84b9a4f24a7f115b, user_id=fc95d77e50fb48e09224a91e47f29feb, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:35.947235 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e578522-ba0c-4ad5-87b8-694fed68ee42 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:35.960506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 831/1629] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:35 2022] PUT /identity/v3/projects/a4ed567f14174818a2e631956d791e87/users/56c24c9641a24308afd2de9cb337b415/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:35.962396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567b1a4-70d8-4226-9f84-f873f95c1dca None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:35.962867 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567b1a4-70d8-4226-9f84-f873f95c1dca None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:35.963268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567b1a4-70d8-4226-9f84-f873f95c1dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:35.991243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4567b1a4-70d8-4226-9f84-f873f95c1dca None None] MFA Rules not processed for user `2a4284b59f804dc59674d621d87634bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:35.993164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4567b1a4-70d8-4226-9f84-f873f95c1dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:36.059849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 799/1630] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:35 2022] PUT /identity/v3/projects/034ba9252ef549ad84b9a4f24a7f115b/users/fc95d77e50fb48e09224a91e47f29feb/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:36.059849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342b8a6-b9ea-4f9b-8f4f-93af28d31081 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:36.059849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342b8a6-b9ea-4f9b-8f4f-93af28d31081 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:36.059849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342b8a6-b9ea-4f9b-8f4f-93af28d31081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:36.085041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 832/1631] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 19:55:35 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:36.086287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567b1a4-70d8-4226-9f84-f873f95c1dca None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:36.123971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-0342b8a6-b9ea-4f9b-8f4f-93af28d31081 None None] MFA Rules not processed for user `56c24c9641a24308afd2de9cb337b415`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:36.127691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0342b8a6-b9ea-4f9b-8f4f-93af28d31081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:36.190127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567b1a4-70d8-4226-9f84-f873f95c1dca None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:36.241053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:36.283593 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606dc82-2e8d-42d6-b109-20d7283459d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:36.284185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606dc82-2e8d-42d6-b109-20d7283459d2 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:36.284462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606dc82-2e8d-42d6-b109-20d7283459d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:36.284736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606dc82-2e8d-42d6-b109-20d7283459d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:36.285382 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606dc82-2e8d-42d6-b109-20d7283459d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:36.287303 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606dc82-2e8d-42d6-b109-20d7283459d2 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:36.300859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 833/1632] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:36.307691 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606dc82-2e8d-42d6-b109-20d7283459d2 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:36.307861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 800/1633] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:36 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:36.308226 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342b8a6-b9ea-4f9b-8f4f-93af28d31081 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:36.331069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342b8a6-b9ea-4f9b-8f4f-93af28d31081 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:36.339993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41363640-3133-4d6a-81f5-d9e886677d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:36.340699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41363640-3133-4d6a-81f5-d9e886677d3b admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:36.341122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41363640-3133-4d6a-81f5-d9e886677d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:36.341225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41363640-3133-4d6a-81f5-d9e886677d3b admin admin] PATH_INFO: `/v3/projects/034ba9252ef549ad84b9a4f24a7f115b/users/fc95d77e50fb48e09224a91e47f29feb/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:36.343343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41363640-3133-4d6a-81f5-d9e886677d3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=034ba9252ef549ad84b9a4f24a7f115b, user_id=fc95d77e50fb48e09224a91e47f29feb, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:36.346452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41363640-3133-4d6a-81f5-d9e886677d3b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:36.376726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 801/1634] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:36 2022] PUT /identity/v3/projects/034ba9252ef549ad84b9a4f24a7f115b/users/fc95d77e50fb48e09224a91e47f29feb/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:36.378356 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690d5fb-28a6-47e9-ace7-960be32aaa9e None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:36.378640 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690d5fb-28a6-47e9-ace7-960be32aaa9e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:36.378964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690d5fb-28a6-47e9-ace7-960be32aaa9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:36.416400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606dc82-2e8d-42d6-b109-20d7283459d2 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:36.436150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-d690d5fb-28a6-47e9-ace7-960be32aaa9e None None] MFA Rules not processed for user `fc95d77e50fb48e09224a91e47f29feb`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:36.438611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d690d5fb-28a6-47e9-ace7-960be32aaa9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:36.485054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8ad37-de99-47cc-a0f3-47bcefd5680d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:36.485723 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8ad37-de99-47cc-a0f3-47bcefd5680d service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:36.486032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8ad37-de99-47cc-a0f3-47bcefd5680d service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:36.486296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8ad37-de99-47cc-a0f3-47bcefd5680d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:36.540193 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 802/1635] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:36.563864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:36.636201 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690d5fb-28a6-47e9-ace7-960be32aaa9e None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:36.641518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8ad37-de99-47cc-a0f3-47bcefd5680d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:36.643011 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8ad37-de99-47cc-a0f3-47bcefd5680d service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:36.650369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d690d5fb-28a6-47e9-ace7-960be32aaa9e None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:36.656771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17986f8f-dfdd-4652-bc6a-f0364e177a2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:36.657597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17986f8f-dfdd-4652-bc6a-f0364e177a2f service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:36.658111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17986f8f-dfdd-4652-bc6a-f0364e177a2f service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:36.658794 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 834/1636] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:36.660619 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17986f8f-dfdd-4652-bc6a-f0364e177a2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:36.745518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17986f8f-dfdd-4652-bc6a-f0364e177a2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:36.747976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17986f8f-dfdd-4652-bc6a-f0364e177a2f service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:36.762680 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 803/1637] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:36 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:36.872066 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8ad37-de99-47cc-a0f3-47bcefd5680d service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:36.880847 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8ad37-de99-47cc-a0f3-47bcefd5680d service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:36.890997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86c338-1b5d-4278-b318-efbbc55ff85a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:36.891808 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86c338-1b5d-4278-b318-efbbc55ff85a service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:36.892177 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86c338-1b5d-4278-b318-efbbc55ff85a service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:36.892644 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef86c338-1b5d-4278-b318-efbbc55ff85a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:36.920871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86c338-1b5d-4278-b318-efbbc55ff85a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:36.926669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef86c338-1b5d-4278-b318-efbbc55ff85a service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:36.944718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 835/1638] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:37.196248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17986f8f-dfdd-4652-bc6a-f0364e177a2f service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:37.223125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17986f8f-dfdd-4652-bc6a-f0364e177a2f service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:37.230318 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdb2bd-cce1-46c8-9372-7ad2717b91db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:37.231755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdb2bd-cce1-46c8-9372-7ad2717b91db service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:37.235512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdb2bd-cce1-46c8-9372-7ad2717b91db service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:37.235512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdb2bd-cce1-46c8-9372-7ad2717b91db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:37.304084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bdb2bd-cce1-46c8-9372-7ad2717b91db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:37.306370 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bdb2bd-cce1-46c8-9372-7ad2717b91db service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:37.317597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 804/1639] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:37 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:37.411656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:37.720834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:38.370269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86c338-1b5d-4278-b318-efbbc55ff85a service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:38.383694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef86c338-1b5d-4278-b318-efbbc55ff85a service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:38.400265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec3b2e9-b778-42f4-bb42-d11d365f4b51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:38.401551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3b2e9-b778-42f4-bb42-d11d365f4b51 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:38.406642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3b2e9-b778-42f4-bb42-d11d365f4b51 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:38.406642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec3b2e9-b778-42f4-bb42-d11d365f4b51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:38.606000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3b2e9-b778-42f4-bb42-d11d365f4b51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:38.609322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec3b2e9-b778-42f4-bb42-d11d365f4b51 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:38.645369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 836/1640] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:39.006219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf6544-60a7-435c-b555-0b10c56c6ebc None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:39.007242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf6544-60a7-435c-b555-0b10c56c6ebc None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:39.007760 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf6544-60a7-435c-b555-0b10c56c6ebc None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:39.009427 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 805/1641] 10.222.0.42 () {54 vars in 962 bytes} [Wed Sep 14 19:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:55:39.018595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d5e98-9020-4ad3-852b-168bd36fbd24 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:39.019732 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d5e98-9020-4ad3-852b-168bd36fbd24 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:39.020216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11d5e98-9020-4ad3-852b-168bd36fbd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:39.085586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f11d5e98-9020-4ad3-852b-168bd36fbd24 None None] MFA Rules not processed for user `2a4284b59f804dc59674d621d87634bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:39.089428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f11d5e98-9020-4ad3-852b-168bd36fbd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:39.111003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 837/1642] 10.222.0.42 () {58 vars in 1059 bytes} [Wed Sep 14 19:55:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:39.119015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd0637-5e8d-47c6-afb4-32b12b53f0c9 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:39.119308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd0637-5e8d-47c6-afb4-32b12b53f0c9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:39.119582 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd0637-5e8d-47c6-afb4-32b12b53f0c9 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:39.121623 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 806/1643] 10.222.0.42 () {54 vars in 912 bytes} [Wed Sep 14 19:55:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:55:39.130696 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f3be4-7f74-4fde-ad71-050b11f9a1d5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:39.131377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f3be4-7f74-4fde-ad71-050b11f9a1d5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:39.131377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023f3be4-7f74-4fde-ad71-050b11f9a1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:39.184453 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-023f3be4-7f74-4fde-ad71-050b11f9a1d5 None None] MFA Rules not processed for user `2a4284b59f804dc59674d621d87634bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:39.185790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-023f3be4-7f74-4fde-ad71-050b11f9a1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:39.212847 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 838/1644] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 19:55:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:39.339315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd0637-5e8d-47c6-afb4-32b12b53f0c9 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:39.357441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd0637-5e8d-47c6-afb4-32b12b53f0c9 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:39.367970 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6807c-2293-4a6b-97ab-503870df6408 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:39.368629 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6807c-2293-4a6b-97ab-503870df6408 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:39.368875 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6807c-2293-4a6b-97ab-503870df6408 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:39.369143 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6807c-2293-4a6b-97ab-503870df6408 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:39.383216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6807c-2293-4a6b-97ab-503870df6408 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:39.384890 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6807c-2293-4a6b-97ab-503870df6408 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:39.399420 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 807/1645] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:46.459673 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00df81e-b633-4db9-bedb-16dfdd4fd983 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:46.460220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00df81e-b633-4db9-bedb-16dfdd4fd983 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:46.460296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00df81e-b633-4db9-bedb-16dfdd4fd983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:46.576879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b00df81e-b633-4db9-bedb-16dfdd4fd983 None None] MFA Rules not processed for user `766dd5a650fc4855980250969854f08d`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:46.587770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b00df81e-b633-4db9-bedb-16dfdd4fd983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:46.817429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 839/1646] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:46.831294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a8c15-b04e-45f7-97a9-3bd3b31ca78a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:46.833073 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a8c15-b04e-45f7-97a9-3bd3b31ca78a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:46.833531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a8c15-b04e-45f7-97a9-3bd3b31ca78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:46.866354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-5d4a8c15-b04e-45f7-97a9-3bd3b31ca78a None None] MFA Rules not processed for user `766dd5a650fc4855980250969854f08d`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:46.869538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-5d4a8c15-b04e-45f7-97a9-3bd3b31ca78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:46.887038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 808/1647] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:46.898663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00df81e-b633-4db9-bedb-16dfdd4fd983 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:46.931260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00df81e-b633-4db9-bedb-16dfdd4fd983 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:46.962218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ce17a-ad89-4659-a9bd-590a5f946bbe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:46.962847 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ce17a-ad89-4659-a9bd-590a5f946bbe service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:46.967371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ce17a-ad89-4659-a9bd-590a5f946bbe service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:46.967718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ce17a-ad89-4659-a9bd-590a5f946bbe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:46.997909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ce17a-ad89-4659-a9bd-590a5f946bbe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:46.999836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ce17a-ad89-4659-a9bd-590a5f946bbe service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:47.037563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a8c15-b04e-45f7-97a9-3bd3b31ca78a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:47.051051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 840/1648] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:47.253714 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a8c15-b04e-45f7-97a9-3bd3b31ca78a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:47.266776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc5852-2baf-4b49-85a7-6b12f8906191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:47.290294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc5852-2baf-4b49-85a7-6b12f8906191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:47.291963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc5852-2baf-4b49-85a7-6b12f8906191 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:47.292447 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fc5852-2baf-4b49-85a7-6b12f8906191 admin admin] PATH_INFO: `/v3/users/ad18d3c6633a47538f29dfd0d2e9db16` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:47.295281 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc5852-2baf-4b49-85a7-6b12f8906191 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad18d3c6633a47538f29dfd0d2e9db16)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:47.304935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fc5852-2baf-4b49-85a7-6b12f8906191 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:47.320439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf05dd-d3b8-4aa6-bce8-7d2d30269ef5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:47.322136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf05dd-d3b8-4aa6-bce8-7d2d30269ef5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:47.322428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf05dd-d3b8-4aa6-bce8-7d2d30269ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:47.423498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-69cf05dd-d3b8-4aa6-bce8-7d2d30269ef5 None None] MFA Rules not processed for user `96428b8e76314536a9e7033a841454e2`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:47.433468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-69cf05dd-d3b8-4aa6-bce8-7d2d30269ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:47.583631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:47.594891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-79fc5852-2baf-4b49-85a7-6b12f8906191 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad18d3c6633a47538f29dfd0d2e9db16'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:47.640328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 841/1649] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:55:47.640462 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-79fc5852-2baf-4b49-85a7-6b12f8906191 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad18d3c6633a47538f29dfd0d2e9db16'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:47.642160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf5e6e-741a-4005-a457-c96a592a230f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:47.642593 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf5e6e-741a-4005-a457-c96a592a230f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:47.643001 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bf5e6e-741a-4005-a457-c96a592a230f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:47.689691 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c0bf5e6e-741a-4005-a457-c96a592a230f None None] MFA Rules not processed for user `96428b8e76314536a9e7033a841454e2`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:47.696746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c0bf5e6e-741a-4005-a457-c96a592a230f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:47.747241 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-79fc5852-2baf-4b49-85a7-6b12f8906191 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad18d3c6633a47538f29dfd0d2e9db16'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:47.748035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-79fc5852-2baf-4b49-85a7-6b12f8906191 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad18d3c6633a47538f29dfd0d2e9db16'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:47.759098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-79fc5852-2baf-4b49-85a7-6b12f8906191 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad18d3c6633a47538f29dfd0d2e9db16'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:47.768238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 842/1650] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:47.778350 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 809/1651] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:55:47 2022] DELETE /identity/v3/users/ad18d3c6633a47538f29dfd0d2e9db16 => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:47.783572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71e84e-65c6-4f23-9c2b-4c32dbec4cee None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:47.783849 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71e84e-65c6-4f23-9c2b-4c32dbec4cee None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:47.784114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71e84e-65c6-4f23-9c2b-4c32dbec4cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:47.831991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-ca71e84e-65c6-4f23-9c2b-4c32dbec4cee None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:47.835521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ca71e84e-65c6-4f23-9c2b-4c32dbec4cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:47.888692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc5852-2baf-4b49-85a7-6b12f8906191 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:47.891112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-79fc5852-2baf-4b49-85a7-6b12f8906191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:48.061267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 843/1652] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:48.087383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:48.106884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71e84e-65c6-4f23-9c2b-4c32dbec4cee None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:48.109104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ca71e84e-65c6-4f23-9c2b-4c32dbec4cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:48.163759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fc5852-2baf-4b49-85a7-6b12f8906191 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:48.183066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af0624b-2265-4777-a1e4-3f4db4470741 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:48.185763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0624b-2265-4777-a1e4-3f4db4470741 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:48.186433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0624b-2265-4777-a1e4-3f4db4470741 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:48.187489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af0624b-2265-4777-a1e4-3f4db4470741 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:48.193229 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-7af0624b-2265-4777-a1e4-3f4db4470741 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:48.259634 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71e84e-65c6-4f23-9c2b-4c32dbec4cee None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:48.271924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be3590-3473-424c-9b04-8d27abd002d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:48.272761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be3590-3473-424c-9b04-8d27abd002d0 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:48.273148 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be3590-3473-424c-9b04-8d27abd002d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:48.273523 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be3590-3473-424c-9b04-8d27abd002d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:48.299693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be3590-3473-424c-9b04-8d27abd002d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:48.301382 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be3590-3473-424c-9b04-8d27abd002d0 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:48.323695 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 844/1653] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:48.333296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a11e74-437d-4a45-912f-65bc7d973e9d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:48.333711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a11e74-437d-4a45-912f-65bc7d973e9d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:48.334157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a11e74-437d-4a45-912f-65bc7d973e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:48.353203 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0624b-2265-4777-a1e4-3f4db4470741 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:48.355815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af0624b-2265-4777-a1e4-3f4db4470741 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:48.398674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 810/1654] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:55:48.402880 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-30a11e74-437d-4a45-912f-65bc7d973e9d None None] MFA Rules not processed for user `2bf1c7a6cc2e4b1294e7b162a8aba738`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:48.404151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-30a11e74-437d-4a45-912f-65bc7d973e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:48.551445 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604f3cf-5bf1-4c39-8cf3-8caa0936e968 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:48.551734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604f3cf-5bf1-4c39-8cf3-8caa0936e968 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:48.552288 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604f3cf-5bf1-4c39-8cf3-8caa0936e968 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:48.555326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 811/1655] 10.222.0.42 () {54 vars in 912 bytes} [Wed Sep 14 19:55:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:55:48.563157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61d369-b308-41f6-962c-57514ba44b0d None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:48.563691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61d369-b308-41f6-962c-57514ba44b0d None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:48.564337 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61d369-b308-41f6-962c-57514ba44b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:48.577681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 845/1656] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:48.678517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-7c61d369-b308-41f6-962c-57514ba44b0d None None] MFA Rules not processed for user `7103ff9544524393a8c7175998dcae3b`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:48.680662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-7c61d369-b308-41f6-962c-57514ba44b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:48.698264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 812/1657] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 19:55:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:55:48.714784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a11e74-437d-4a45-912f-65bc7d973e9d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:48.729345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a11e74-437d-4a45-912f-65bc7d973e9d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:48.733669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ba7f96-ba98-40ea-9880-820e87fa90aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:48.734373 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba7f96-ba98-40ea-9880-820e87fa90aa service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:48.734729 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba7f96-ba98-40ea-9880-820e87fa90aa service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:48.734993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ba7f96-ba98-40ea-9880-820e87fa90aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:48.743413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba7f96-ba98-40ea-9880-820e87fa90aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:48.745145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ba7f96-ba98-40ea-9880-820e87fa90aa service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:48.767902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 846/1658] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:49.166791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a613e-c9d0-4542-8dc9-a760ae449608 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:49.167355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a613e-c9d0-4542-8dc9-a760ae449608 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:49.167644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a613e-c9d0-4542-8dc9-a760ae449608 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:49.169113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 813/1659] 10.222.0.42 () {54 vars in 912 bytes} [Wed Sep 14 19:55:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:55:49.177165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65245ddc-08ba-4513-8173-bdfe48fb61de None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:49.177467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65245ddc-08ba-4513-8173-bdfe48fb61de None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:49.177733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65245ddc-08ba-4513-8173-bdfe48fb61de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:49.195004 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:49.224240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-65245ddc-08ba-4513-8173-bdfe48fb61de None None] MFA Rules not processed for user `7103ff9544524393a8c7175998dcae3b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:49.226693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-65245ddc-08ba-4513-8173-bdfe48fb61de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:49.260078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 847/1660] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 19:55:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:49.277673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a613e-c9d0-4542-8dc9-a760ae449608 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:49.279500 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-433a613e-c9d0-4542-8dc9-a760ae449608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:49.305134 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a613e-c9d0-4542-8dc9-a760ae449608 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:49.324957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe2b1a-84b8-4b86-bbc6-04d9f6277430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:49.325869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe2b1a-84b8-4b86-bbc6-04d9f6277430 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:49.326290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe2b1a-84b8-4b86-bbc6-04d9f6277430 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:49.326701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe2b1a-84b8-4b86-bbc6-04d9f6277430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:49.356916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe2b1a-84b8-4b86-bbc6-04d9f6277430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:49.358471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe2b1a-84b8-4b86-bbc6-04d9f6277430 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:49.385074 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 814/1661] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:49.709252 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65245ddc-08ba-4513-8173-bdfe48fb61de None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:49.710716 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-65245ddc-08ba-4513-8173-bdfe48fb61de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:49.735785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65245ddc-08ba-4513-8173-bdfe48fb61de None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:49.758828 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:49.758828 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:49.758828 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:49.758828 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] PATH_INFO: `/v3/projects/3240927f0b59480d9e218342273b33c9` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:49.759144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3240927f0b59480d9e218342273b33c9)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:49.760009 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:50.021950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3240927f0b59480d9e218342273b33c9', 'request_id': 'req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:50.160348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3240927f0b59480d9e218342273b33c9', 'request_id': 'req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:50.172941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abe246-c5b4-4f68-82f8-f9a6ae0deeb1 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:50.173507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abe246-c5b4-4f68-82f8-f9a6ae0deeb1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:50.173507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abe246-c5b4-4f68-82f8-f9a6ae0deeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:50.183313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3240927f0b59480d9e218342273b33c9', 'request_id': 'req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:50.212893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] The token cache is being invalidate because project 3240927f0b59480d9e218342273b33c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:55:50.214728 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:50.220764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:50.223139 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 848/1662] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:55:49 2022] DELETE /identity/v3/projects/3240927f0b59480d9e218342273b33c9 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:50.227115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:50.230287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:50.272148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b3abe246-c5b4-4f68-82f8-f9a6ae0deeb1 None None] MFA Rules not processed for user `268dbf54f29c40d9a4daa772c44735d4`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:50.273656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b3abe246-c5b4-4f68-82f8-f9a6ae0deeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:50.445105 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e7d42-d318-4d0b-9cab-dea018ed6e30 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:50.479629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4008c1-b5a8-4130-a8ec-3415e47bf386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:50.480288 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4008c1-b5a8-4130-a8ec-3415e47bf386 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:50.480563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4008c1-b5a8-4130-a8ec-3415e47bf386 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:50.480833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4008c1-b5a8-4130-a8ec-3415e47bf386 admin admin] PATH_INFO: `/v3/users/56c24c9641a24308afd2de9cb337b415` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:50.482331 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4008c1-b5a8-4130-a8ec-3415e47bf386 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56c24c9641a24308afd2de9cb337b415)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:50.483357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: WARNING py.warnings [None req-dc4008c1-b5a8-4130-a8ec-3415e47bf386 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:55:50.483357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: warnings.warn(msg) Sep 14 19:55:50.483357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: Sep 14 19:55:50.486567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4008c1-b5a8-4130-a8ec-3415e47bf386 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:50.563659 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 815/1663] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:50.565207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abe246-c5b4-4f68-82f8-f9a6ae0deeb1 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:50.570643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b3abe246-c5b4-4f68-82f8-f9a6ae0deeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:50.677308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:50.785360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abe246-c5b4-4f68-82f8-f9a6ae0deeb1 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:50.798245 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1ef6f-d0db-41ff-8c72-6edaf2fc2bbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:50.803628 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1ef6f-d0db-41ff-8c72-6edaf2fc2bbc service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:50.810156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1ef6f-d0db-41ff-8c72-6edaf2fc2bbc service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:50.810437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1ef6f-d0db-41ff-8c72-6edaf2fc2bbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:50.817105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c8b1ef6f-d0db-41ff-8c72-6edaf2fc2bbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:50.880599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dc4008c1-b5a8-4130-a8ec-3415e47bf386 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56c24c9641a24308afd2de9cb337b415'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:50.934184 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1ef6f-d0db-41ff-8c72-6edaf2fc2bbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:50.935965 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b1ef6f-d0db-41ff-8c72-6edaf2fc2bbc service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:50.959830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 816/1664] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:50 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:50.960869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1ef6f-d0db-41ff-8c72-6edaf2fc2bbc service nova] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:50.962623 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c8b1ef6f-d0db-41ff-8c72-6edaf2fc2bbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:51.028010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dc4008c1-b5a8-4130-a8ec-3415e47bf386 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56c24c9641a24308afd2de9cb337b415'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:51.074933 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dc4008c1-b5a8-4130-a8ec-3415e47bf386 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56c24c9641a24308afd2de9cb337b415'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:51.075787 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dc4008c1-b5a8-4130-a8ec-3415e47bf386 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56c24c9641a24308afd2de9cb337b415'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:51.079099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dc4008c1-b5a8-4130-a8ec-3415e47bf386 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56c24c9641a24308afd2de9cb337b415'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:51.146739 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:51.154840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 849/1665] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:55:50 2022] DELETE /identity/v3/users/56c24c9641a24308afd2de9cb337b415 => generated 0 bytes in 928 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:51.158844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d466720-31d2-4840-8daa-3ddc26e69d5b None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:51.159290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d466720-31d2-4840-8daa-3ddc26e69d5b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:51.163335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d466720-31d2-4840-8daa-3ddc26e69d5b None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:51.175159 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 850/1666] 10.222.0.42 () {54 vars in 961 bytes} [Wed Sep 14 19:55:51 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:55:51.177768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07281928-6610-4bcc-b275-6f9ebc4e5662 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:51.180689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07281928-6610-4bcc-b275-6f9ebc4e5662 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:51.181592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07281928-6610-4bcc-b275-6f9ebc4e5662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:51.196144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1ef6f-d0db-41ff-8c72-6edaf2fc2bbc service nova] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:51.217626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-07281928-6610-4bcc-b275-6f9ebc4e5662 None None] MFA Rules not processed for user `766dd5a650fc4855980250969854f08d`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:51.220706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-07281928-6610-4bcc-b275-6f9ebc4e5662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:51.350916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 851/1667] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:51.356501 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07281928-6610-4bcc-b275-6f9ebc4e5662 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:51.358696 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-07281928-6610-4bcc-b275-6f9ebc4e5662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:51.389317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f203f1c-8edb-404d-bda2-5f3853ba8edb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03bc280d0395479981b7c7b08c96001b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:51.392734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f203f1c-8edb-404d-bda2-5f3853ba8edb service placement] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:51.392734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f203f1c-8edb-404d-bda2-5f3853ba8edb service placement] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:51.392734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f203f1c-8edb-404d-bda2-5f3853ba8edb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:51.405760 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4f203f1c-8edb-404d-bda2-5f3853ba8edb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:51.439016 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f203f1c-8edb-404d-bda2-5f3853ba8edb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:51.442835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f203f1c-8edb-404d-bda2-5f3853ba8edb service placement] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:51.472212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 817/1668] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:51.475366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7625fa-e5f2-454d-a238-91fecbd81f54 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:51.475819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7625fa-e5f2-454d-a238-91fecbd81f54 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:51.477327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7625fa-e5f2-454d-a238-91fecbd81f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:51.524631 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-bd7625fa-e5f2-454d-a238-91fecbd81f54 None None] MFA Rules not processed for user `03bc280d0395479981b7c7b08c96001b`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:51.528591 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bd7625fa-e5f2-454d-a238-91fecbd81f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:51.564169 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 818/1669] 10.222.0.42 () {58 vars in 1058 bytes} [Wed Sep 14 19:55:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:51.570463 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7625fa-e5f2-454d-a238-91fecbd81f54 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:51.572720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bd7625fa-e5f2-454d-a238-91fecbd81f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:51.689685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07281928-6610-4bcc-b275-6f9ebc4e5662 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:51.746838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb2607-0ad4-40c3-95d4-b5d0f1dabace service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:51.748919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb2607-0ad4-40c3-95d4-b5d0f1dabace service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:51.758387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb2607-0ad4-40c3-95d4-b5d0f1dabace service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:51.758387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb2607-0ad4-40c3-95d4-b5d0f1dabace service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:51.768769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-24bb2607-0ad4-40c3-95d4-b5d0f1dabace service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:51.809123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:51.884807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7625fa-e5f2-454d-a238-91fecbd81f54 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:51.915674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d2a1c-d68c-4ede-9b9d-f7f4cbbfc875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:51.916396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d2a1c-d68c-4ede-9b9d-f7f4cbbfc875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:51.916676 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d2a1c-d68c-4ede-9b9d-f7f4cbbfc875 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:51.916942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d2a1c-d68c-4ede-9b9d-f7f4cbbfc875 admin admin] PATH_INFO: `/v3/users/01d6926769014a88bbfd4be361b2a307` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:51.931801 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d2a1c-d68c-4ede-9b9d-f7f4cbbfc875 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01d6926769014a88bbfd4be361b2a307)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:51.938156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d2a1c-d68c-4ede-9b9d-f7f4cbbfc875 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:52.043323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb2607-0ad4-40c3-95d4-b5d0f1dabace service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:52.048538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb2607-0ad4-40c3-95d4-b5d0f1dabace service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:52.062068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 852/1670] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:52.063324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb2607-0ad4-40c3-95d4-b5d0f1dabace service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:52.077215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb2607-0ad4-40c3-95d4-b5d0f1dabace service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:52.084205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b04f09-9d1c-4614-9d0d-c5ea33180503 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03bc280d0395479981b7c7b08c96001b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:52.085119 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b04f09-9d1c-4614-9d0d-c5ea33180503 service placement] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:52.085582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b04f09-9d1c-4614-9d0d-c5ea33180503 service placement] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:52.086305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b04f09-9d1c-4614-9d0d-c5ea33180503 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:52.105662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b04f09-9d1c-4614-9d0d-c5ea33180503 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:52.107621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b04f09-9d1c-4614-9d0d-c5ea33180503 service placement] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:52.137546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 853/1671] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:52.286531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8e2d2a1c-d68c-4ede-9b9d-f7f4cbbfc875 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01d6926769014a88bbfd4be361b2a307'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:52.348702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8e2d2a1c-d68c-4ede-9b9d-f7f4cbbfc875 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01d6926769014a88bbfd4be361b2a307'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:52.526177 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8e2d2a1c-d68c-4ede-9b9d-f7f4cbbfc875 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01d6926769014a88bbfd4be361b2a307'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:52.526808 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8e2d2a1c-d68c-4ede-9b9d-f7f4cbbfc875 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01d6926769014a88bbfd4be361b2a307'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:52.535485 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8e2d2a1c-d68c-4ede-9b9d-f7f4cbbfc875 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01d6926769014a88bbfd4be361b2a307'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:52.554844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 819/1672] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:55:51 2022] DELETE /identity/v3/users/01d6926769014a88bbfd4be361b2a307 => generated 0 bytes in 990 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:52.559248 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64879e77-063f-4d83-beb0-d2968da2e44c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:52.559865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64879e77-063f-4d83-beb0-d2968da2e44c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:52.560371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64879e77-063f-4d83-beb0-d2968da2e44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:52.611668 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-64879e77-063f-4d83-beb0-d2968da2e44c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:52.618693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-64879e77-063f-4d83-beb0-d2968da2e44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:52.855588 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 854/1673] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:53.072432 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d2a1c-d68c-4ede-9b9d-f7f4cbbfc875 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:53.073983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8e2d2a1c-d68c-4ede-9b9d-f7f4cbbfc875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:53.080461 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64879e77-063f-4d83-beb0-d2968da2e44c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:53.081908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-64879e77-063f-4d83-beb0-d2968da2e44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:53.114308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d2a1c-d68c-4ede-9b9d-f7f4cbbfc875 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:53.128423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5079c376-03cb-4873-8fda-1f2e99684ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:53.129077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079c376-03cb-4873-8fda-1f2e99684ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:53.129544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079c376-03cb-4873-8fda-1f2e99684ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:53.129986 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5079c376-03cb-4873-8fda-1f2e99684ee3 admin admin] PATH_INFO: `/v3/projects/a4ed567f14174818a2e631956d791e87` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:53.133626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079c376-03cb-4873-8fda-1f2e99684ee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4ed567f14174818a2e631956d791e87)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:53.134870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: WARNING py.warnings [None req-5079c376-03cb-4873-8fda-1f2e99684ee3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 14 19:55:53.134870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: warnings.warn(msg) Sep 14 19:55:53.134870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: Sep 14 19:55:53.141529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5079c376-03cb-4873-8fda-1f2e99684ee3 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:53.211444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64879e77-063f-4d83-beb0-d2968da2e44c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:53.216868 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e630a0-209a-4b8e-80b3-6b250a5e0476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:53.218205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e630a0-209a-4b8e-80b3-6b250a5e0476 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:53.218494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e630a0-209a-4b8e-80b3-6b250a5e0476 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:53.219069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e630a0-209a-4b8e-80b3-6b250a5e0476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:53.256664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e630a0-209a-4b8e-80b3-6b250a5e0476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:53.259432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e630a0-209a-4b8e-80b3-6b250a5e0476 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:53.283798 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 855/1674] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:55:53.356225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5079c376-03cb-4873-8fda-1f2e99684ee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4ed567f14174818a2e631956d791e87', 'request_id': 'req-5079c376-03cb-4873-8fda-1f2e99684ee3'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:53.362837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5079c376-03cb-4873-8fda-1f2e99684ee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4ed567f14174818a2e631956d791e87', 'request_id': 'req-5079c376-03cb-4873-8fda-1f2e99684ee3'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:53.383618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5079c376-03cb-4873-8fda-1f2e99684ee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4ed567f14174818a2e631956d791e87', 'request_id': 'req-5079c376-03cb-4873-8fda-1f2e99684ee3'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:53.398549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e630a0-209a-4b8e-80b3-6b250a5e0476 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:53.400632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f2e630a0-209a-4b8e-80b3-6b250a5e0476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:53.418493 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e630a0-209a-4b8e-80b3-6b250a5e0476 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:53.522225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5079c376-03cb-4873-8fda-1f2e99684ee3 admin admin] The token cache is being invalidate because project a4ed567f14174818a2e631956d791e87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:55:53.522658 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5079c376-03cb-4873-8fda-1f2e99684ee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:53.523971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5079c376-03cb-4873-8fda-1f2e99684ee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:53.528139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 820/1675] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:55:53 2022] DELETE /identity/v3/projects/a4ed567f14174818a2e631956d791e87 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:53.531202 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89589b0f-17d3-4814-8c44-f7c5e03d916c None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:53.531684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89589b0f-17d3-4814-8c44-f7c5e03d916c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:53.532151 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89589b0f-17d3-4814-8c44-f7c5e03d916c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:53.545833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:53.574068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-89589b0f-17d3-4814-8c44-f7c5e03d916c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:53.575646 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-89589b0f-17d3-4814-8c44-f7c5e03d916c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:53.581277 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838ee6c4-d129-4fa8-a896-e12aea44f4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:53.582013 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ee6c4-d129-4fa8-a896-e12aea44f4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:53.584941 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ee6c4-d129-4fa8-a896-e12aea44f4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:53.585246 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838ee6c4-d129-4fa8-a896-e12aea44f4ba admin admin] PATH_INFO: `/v3/users/0e73f6986ae247369ab904edea2b51a0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:53.592289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ee6c4-d129-4fa8-a896-e12aea44f4ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e73f6986ae247369ab904edea2b51a0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:53.597141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838ee6c4-d129-4fa8-a896-e12aea44f4ba admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:53.599081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 821/1676] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:53.610105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4f3e-a62d-41f4-827d-093ad5f9b0ea None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:53.610590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4f3e-a62d-41f4-827d-093ad5f9b0ea None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:53.610885 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f4f3e-a62d-41f4-827d-093ad5f9b0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:53.657544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b39f4f3e-a62d-41f4-827d-093ad5f9b0ea None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:53.659785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b39f4f3e-a62d-41f4-827d-093ad5f9b0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:53.670971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 822/1677] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:53.675264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f4f3e-a62d-41f4-827d-093ad5f9b0ea None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:53.689407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f4f3e-a62d-41f4-827d-093ad5f9b0ea None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:53.699009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242365d9-acba-4ce6-8e3d-307022a9d2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:53.699859 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242365d9-acba-4ce6-8e3d-307022a9d2ce admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:53.700197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242365d9-acba-4ce6-8e3d-307022a9d2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:53.700504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242365d9-acba-4ce6-8e3d-307022a9d2ce admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:53.701139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242365d9-acba-4ce6-8e3d-307022a9d2ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:55:53.701442 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242365d9-acba-4ce6-8e3d-307022a9d2ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:53.703997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242365d9-acba-4ce6-8e3d-307022a9d2ce admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:53.723348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 823/1678] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:55:53.727540 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242365d9-acba-4ce6-8e3d-307022a9d2ce admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:53.761920 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242365d9-acba-4ce6-8e3d-307022a9d2ce admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:53.771751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907c47a-00d7-4866-aa8b-19dce5178d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:53.772518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907c47a-00d7-4866-aa8b-19dce5178d24 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:53.772875 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907c47a-00d7-4866-aa8b-19dce5178d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:53.773212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907c47a-00d7-4866-aa8b-19dce5178d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:53.774184 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907c47a-00d7-4866-aa8b-19dce5178d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:53.776303 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907c47a-00d7-4866-aa8b-19dce5178d24 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:53.879675 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-838ee6c4-d129-4fa8-a896-e12aea44f4ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e73f6986ae247369ab904edea2b51a0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:53.954159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 824/1679] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:55:53 2022] POST /identity/v3/projects => generated 401 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:53.956011 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907c47a-00d7-4866-aa8b-19dce5178d24 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:54.069727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907c47a-00d7-4866-aa8b-19dce5178d24 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:54.079979 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db6118-f64d-43e8-bfe3-57df2a95a68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:54.080788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db6118-f64d-43e8-bfe3-57df2a95a68f admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:54.081202 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db6118-f64d-43e8-bfe3-57df2a95a68f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:54.081483 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db6118-f64d-43e8-bfe3-57df2a95a68f admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:54.082150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db6118-f64d-43e8-bfe3-57df2a95a68f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:54.084636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db6118-f64d-43e8-bfe3-57df2a95a68f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:54.094189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-838ee6c4-d129-4fa8-a896-e12aea44f4ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e73f6986ae247369ab904edea2b51a0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:54.137863 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-838ee6c4-d129-4fa8-a896-e12aea44f4ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e73f6986ae247369ab904edea2b51a0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:54.140599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-838ee6c4-d129-4fa8-a896-e12aea44f4ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e73f6986ae247369ab904edea2b51a0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:54.142101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-838ee6c4-d129-4fa8-a896-e12aea44f4ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e73f6986ae247369ab904edea2b51a0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:54.175453 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 856/1680] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:55:53 2022] DELETE /identity/v3/users/0e73f6986ae247369ab904edea2b51a0 => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:55:54.181685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c46c2-540e-4307-a525-6d581baa3928 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:54.192254 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c46c2-540e-4307-a525-6d581baa3928 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:54.197373 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c46c2-540e-4307-a525-6d581baa3928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:54.221265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 825/1681] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:55:53 2022] POST /identity/v3/users => generated 301 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:54.226108 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db6118-f64d-43e8-bfe3-57df2a95a68f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:54.228989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-14db6118-f64d-43e8-bfe3-57df2a95a68f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:54.272278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db6118-f64d-43e8-bfe3-57df2a95a68f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:54.285264 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b17c46c2-540e-4307-a525-6d581baa3928 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:54.293132 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b17c46c2-540e-4307-a525-6d581baa3928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:54.457786 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:54.492873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525d6ea-3e1b-4b6b-a635-37c34f21be45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:54.493583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525d6ea-3e1b-4b6b-a635-37c34f21be45 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:54.495127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525d6ea-3e1b-4b6b-a635-37c34f21be45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:54.495127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8525d6ea-3e1b-4b6b-a635-37c34f21be45 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:54.495127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525d6ea-3e1b-4b6b-a635-37c34f21be45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:54.496315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8525d6ea-3e1b-4b6b-a635-37c34f21be45 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:54.525301 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 857/1682] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:54.568719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 826/1683] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:54.572922 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c46c2-540e-4307-a525-6d581baa3928 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:54.594049 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c46c2-540e-4307-a525-6d581baa3928 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:54.605603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afeca4-a25b-421a-ae76-70b3a4873f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:54.607248 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afeca4-a25b-421a-ae76-70b3a4873f5f admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:54.612397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afeca4-a25b-421a-ae76-70b3a4873f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:54.612397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afeca4-a25b-421a-ae76-70b3a4873f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:54.612397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afeca4-a25b-421a-ae76-70b3a4873f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:54.613851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afeca4-a25b-421a-ae76-70b3a4873f5f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:54.638653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 858/1684] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:54.643506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525d6ea-3e1b-4b6b-a635-37c34f21be45 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:54.652007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afeca4-a25b-421a-ae76-70b3a4873f5f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:54.660180 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-62afeca4-a25b-421a-ae76-70b3a4873f5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:54.663529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8525d6ea-3e1b-4b6b-a635-37c34f21be45 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:54.685885 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb5cef-14cf-402e-a643-b170250813f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:54.686583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb5cef-14cf-402e-a643-b170250813f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:54.686846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb5cef-14cf-402e-a643-b170250813f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:54.687123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb5cef-14cf-402e-a643-b170250813f7 admin admin] PATH_INFO: `/v3/projects/e0f3036a64644244a3b261c9ec408170/users/9b5a572ff16844a795f44f486cb3cb4e/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:54.741861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb5cef-14cf-402e-a643-b170250813f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f3036a64644244a3b261c9ec408170, user_id=9b5a572ff16844a795f44f486cb3cb4e, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:54.758363 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb5cef-14cf-402e-a643-b170250813f7 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:54.867169 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afeca4-a25b-421a-ae76-70b3a4873f5f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:54.872467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b95eb-2e41-49a1-a29e-c533c54d2dc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:54.875238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b95eb-2e41-49a1-a29e-c533c54d2dc3 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:54.875802 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b95eb-2e41-49a1-a29e-c533c54d2dc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:54.876135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b95eb-2e41-49a1-a29e-c533c54d2dc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:54.895995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 827/1685] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:54 2022] PUT /identity/v3/projects/e0f3036a64644244a3b261c9ec408170/users/9b5a572ff16844a795f44f486cb3cb4e/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:54.908619 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dcc57-7cc0-49ab-8ef2-68b0609455ec None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:54.909386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dcc57-7cc0-49ab-8ef2-68b0609455ec None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:54.909386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dcc57-7cc0-49ab-8ef2-68b0609455ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:54.923489 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b95eb-2e41-49a1-a29e-c533c54d2dc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:54.929665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b95eb-2e41-49a1-a29e-c533c54d2dc3 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:54.978569 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 859/1686] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:55.006821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-497dcc57-7cc0-49ab-8ef2-68b0609455ec None None] MFA Rules not processed for user `9b5a572ff16844a795f44f486cb3cb4e`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:55.008117 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-497dcc57-7cc0-49ab-8ef2-68b0609455ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:55.226146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 828/1687] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:54 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:55:55.229540 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b95eb-2e41-49a1-a29e-c533c54d2dc3 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:55.258084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b95eb-2e41-49a1-a29e-c533c54d2dc3 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:55.487865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49368c5b-e7ea-431b-8f0f-349e74427663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:55.489238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49368c5b-e7ea-431b-8f0f-349e74427663 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:55.490227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49368c5b-e7ea-431b-8f0f-349e74427663 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:55.492885 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49368c5b-e7ea-431b-8f0f-349e74427663 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:55.493645 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49368c5b-e7ea-431b-8f0f-349e74427663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:55.507729 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:55.513949 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49368c5b-e7ea-431b-8f0f-349e74427663 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:55.579045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dcc57-7cc0-49ab-8ef2-68b0609455ec None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:55.580018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-497dcc57-7cc0-49ab-8ef2-68b0609455ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:55.598597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dcc57-7cc0-49ab-8ef2-68b0609455ec None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:55.603764 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69daa5c8-706b-4493-9451-51c9852a2dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:55.607113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daa5c8-706b-4493-9451-51c9852a2dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:55.607396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daa5c8-706b-4493-9451-51c9852a2dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:55.613646 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69daa5c8-706b-4493-9451-51c9852a2dc3 admin admin] PATH_INFO: `/v3/projects/98ab5a8a92a04992a4ad32e96f32306c` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:55.631290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69daa5c8-706b-4493-9451-51c9852a2dc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98ab5a8a92a04992a4ad32e96f32306c)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:55.638922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69daa5c8-706b-4493-9451-51c9852a2dc3 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:55.716272 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 860/1688] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:55:55 2022] POST /identity/v3/projects => generated 399 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:55.716962 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49368c5b-e7ea-431b-8f0f-349e74427663 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:55.864936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-69daa5c8-706b-4493-9451-51c9852a2dc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98ab5a8a92a04992a4ad32e96f32306c', 'request_id': 'req-69daa5c8-706b-4493-9451-51c9852a2dc3'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:55.885405 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-69daa5c8-706b-4493-9451-51c9852a2dc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98ab5a8a92a04992a4ad32e96f32306c', 'request_id': 'req-69daa5c8-706b-4493-9451-51c9852a2dc3'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:55.900596 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-69daa5c8-706b-4493-9451-51c9852a2dc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98ab5a8a92a04992a4ad32e96f32306c', 'request_id': 'req-69daa5c8-706b-4493-9451-51c9852a2dc3'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:55.918964 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49368c5b-e7ea-431b-8f0f-349e74427663 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:56.031298 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-69daa5c8-706b-4493-9451-51c9852a2dc3 admin admin] The token cache is being invalidate because project 98ab5a8a92a04992a4ad32e96f32306c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:55:56.032116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-69daa5c8-706b-4493-9451-51c9852a2dc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:56.033002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-69daa5c8-706b-4493-9451-51c9852a2dc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:56.037860 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 829/1689] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:55:55 2022] DELETE /identity/v3/projects/98ab5a8a92a04992a4ad32e96f32306c => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:56.041238 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee03759-f6fb-4ab6-96a0-9463d1cd6e49 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:56.042088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee03759-f6fb-4ab6-96a0-9463d1cd6e49 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:56.042371 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee03759-f6fb-4ab6-96a0-9463d1cd6e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:56.048484 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67aebde-425d-4070-8948-af7f3e89a139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:56.049514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67aebde-425d-4070-8948-af7f3e89a139 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:56.050333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67aebde-425d-4070-8948-af7f3e89a139 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:56.050787 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67aebde-425d-4070-8948-af7f3e89a139 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:56.051866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67aebde-425d-4070-8948-af7f3e89a139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:56.055765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67aebde-425d-4070-8948-af7f3e89a139 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:56.069227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-eee03759-f6fb-4ab6-96a0-9463d1cd6e49 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:56.070880 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-eee03759-f6fb-4ab6-96a0-9463d1cd6e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:56.088159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 830/1690] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:56.096386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94217f71-aa79-454c-a5ea-b2da737f5ee1 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:56.096756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94217f71-aa79-454c-a5ea-b2da737f5ee1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:56.097035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94217f71-aa79-454c-a5ea-b2da737f5ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:56.140796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-94217f71-aa79-454c-a5ea-b2da737f5ee1 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:56.142369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-94217f71-aa79-454c-a5ea-b2da737f5ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:56.180309 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 831/1691] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:55:56.185248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94217f71-aa79-454c-a5ea-b2da737f5ee1 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:56.201483 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94217f71-aa79-454c-a5ea-b2da737f5ee1 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:56.217367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d1165-cf74-421b-b9ed-d65898a38e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:56.218132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d1165-cf74-421b-b9ed-d65898a38e57 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:56.218524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d1165-cf74-421b-b9ed-d65898a38e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:56.218865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d1165-cf74-421b-b9ed-d65898a38e57 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:56.219641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d1165-cf74-421b-b9ed-d65898a38e57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:55:56.219939 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d1165-cf74-421b-b9ed-d65898a38e57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:56.223208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d1165-cf74-421b-b9ed-d65898a38e57 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:56.246850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 861/1692] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:55:55 2022] POST /identity/v3/users => generated 306 bytes in 530 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:56.251490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67aebde-425d-4070-8948-af7f3e89a139 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:56.252520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 832/1693] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:55:56.253849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d1165-cf74-421b-b9ed-d65898a38e57 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:56.254988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a67aebde-425d-4070-8948-af7f3e89a139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:56.278500 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d1165-cf74-421b-b9ed-d65898a38e57 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:56.282585 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67aebde-425d-4070-8948-af7f3e89a139 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:56.289528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142ad42-db72-410b-a955-3bdd0c726c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:56.290153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142ad42-db72-410b-a955-3bdd0c726c56 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:56.290153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142ad42-db72-410b-a955-3bdd0c726c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:56.290367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4142ad42-db72-410b-a955-3bdd0c726c56 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:56.291041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142ad42-db72-410b-a955-3bdd0c726c56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:56.299694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4142ad42-db72-410b-a955-3bdd0c726c56 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:56.302350 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d69512-4abd-4416-bf57-fff637f37e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:56.302984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d69512-4abd-4416-bf57-fff637f37e04 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:56.303264 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d69512-4abd-4416-bf57-fff637f37e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:56.303540 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d69512-4abd-4416-bf57-fff637f37e04 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:56.304185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d69512-4abd-4416-bf57-fff637f37e04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:56.305977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d69512-4abd-4416-bf57-fff637f37e04 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:56.336980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 862/1694] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:56.339197 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d69512-4abd-4416-bf57-fff637f37e04 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:56.363441 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d69512-4abd-4416-bf57-fff637f37e04 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:56.402275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d46a11-ffab-4a23-ba78-8a35338c7342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:56.402903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d46a11-ffab-4a23-ba78-8a35338c7342 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:56.403188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d46a11-ffab-4a23-ba78-8a35338c7342 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:56.404734 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d46a11-ffab-4a23-ba78-8a35338c7342 admin admin] PATH_INFO: `/v3/projects/32193b443b1443de8699120f5f564840/users/664cd13a536d49d387d96c53e40776f4/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:56.434510 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d46a11-ffab-4a23-ba78-8a35338c7342 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32193b443b1443de8699120f5f564840, user_id=664cd13a536d49d387d96c53e40776f4, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:56.438758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 833/1695] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:55:56 2022] POST /identity/v3/projects => generated 373 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:56.438914 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d46a11-ffab-4a23-ba78-8a35338c7342 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:56.439787 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142ad42-db72-410b-a955-3bdd0c726c56 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:56.518206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4142ad42-db72-410b-a955-3bdd0c726c56 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:56.524127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7a077-1616-4771-80f7-9749be3aecd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:56.524849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7a077-1616-4771-80f7-9749be3aecd0 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:56.532012 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7a077-1616-4771-80f7-9749be3aecd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:56.532012 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7a077-1616-4771-80f7-9749be3aecd0 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:56.532012 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7a077-1616-4771-80f7-9749be3aecd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:56.532012 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7a077-1616-4771-80f7-9749be3aecd0 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:56.561864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 863/1696] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:56 2022] PUT /identity/v3/projects/32193b443b1443de8699120f5f564840/users/664cd13a536d49d387d96c53e40776f4/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:56.567145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d46a11-ffab-4a23-ba78-8a35338c7342 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:56.677566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 834/1697] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:55:56 2022] POST /identity/v3/users => generated 287 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:55:56.681484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7a077-1616-4771-80f7-9749be3aecd0 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:56.695202 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a7a7a077-1616-4771-80f7-9749be3aecd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:56.707596 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d46a11-ffab-4a23-ba78-8a35338c7342 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:56.720860 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1734d-77d5-43df-a46f-501aacde0fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:56.721495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1734d-77d5-43df-a46f-501aacde0fbe admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:56.724110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1734d-77d5-43df-a46f-501aacde0fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:56.725632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1734d-77d5-43df-a46f-501aacde0fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:56.726442 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1734d-77d5-43df-a46f-501aacde0fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:56.731075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:56.733840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1734d-77d5-43df-a46f-501aacde0fbe admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:56.733981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7a077-1616-4771-80f7-9749be3aecd0 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:56.759315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4045fc-7125-462d-9214-63e546b536f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:56.759985 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4045fc-7125-462d-9214-63e546b536f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:56.760255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4045fc-7125-462d-9214-63e546b536f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:56.760527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4045fc-7125-462d-9214-63e546b536f0 admin admin] PATH_INFO: `/v3/projects/28a7633a75a745049c541513cd4b1120` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:56.766745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4045fc-7125-462d-9214-63e546b536f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28a7633a75a745049c541513cd4b1120)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:56.773121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 864/1698] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:55:56.773301 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4045fc-7125-462d-9214-63e546b536f0 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:56.776747 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1734d-77d5-43df-a46f-501aacde0fbe admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:56.817276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1734d-77d5-43df-a46f-501aacde0fbe admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:56.830659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c0f2e-fece-48f2-8e84-f3e99ac80f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:56.831319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c0f2e-fece-48f2-8e84-f3e99ac80f26 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:56.831807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c0f2e-fece-48f2-8e84-f3e99ac80f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:56.832055 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9c0f2e-fece-48f2-8e84-f3e99ac80f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:56.832692 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c0f2e-fece-48f2-8e84-f3e99ac80f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:56.834821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9c0f2e-fece-48f2-8e84-f3e99ac80f26 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:56.856074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 865/1699] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:56.858715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c0f2e-fece-48f2-8e84-f3e99ac80f26 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:56.877459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9c0f2e-fece-48f2-8e84-f3e99ac80f26 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:56.882766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e443575-5df0-45cd-9aed-ebf8c4f6f012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:56.883447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e443575-5df0-45cd-9aed-ebf8c4f6f012 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:56.883727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e443575-5df0-45cd-9aed-ebf8c4f6f012 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:56.884002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e443575-5df0-45cd-9aed-ebf8c4f6f012 admin admin] PATH_INFO: `/v3/projects/32193b443b1443de8699120f5f564840/users/664cd13a536d49d387d96c53e40776f4/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:56.887023 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e443575-5df0-45cd-9aed-ebf8c4f6f012 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32193b443b1443de8699120f5f564840, user_id=664cd13a536d49d387d96c53e40776f4, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:56.890900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e443575-5df0-45cd-9aed-ebf8c4f6f012 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:56.918781 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 866/1700] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:56 2022] PUT /identity/v3/projects/32193b443b1443de8699120f5f564840/users/664cd13a536d49d387d96c53e40776f4/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:56.920007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e443575-5df0-45cd-9aed-ebf8c4f6f012 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:56.958734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-7b4045fc-7125-462d-9214-63e546b536f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28a7633a75a745049c541513cd4b1120', 'request_id': 'req-7b4045fc-7125-462d-9214-63e546b536f0'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:56.972253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-7b4045fc-7125-462d-9214-63e546b536f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28a7633a75a745049c541513cd4b1120', 'request_id': 'req-7b4045fc-7125-462d-9214-63e546b536f0'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:56.987046 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-7b4045fc-7125-462d-9214-63e546b536f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28a7633a75a745049c541513cd4b1120', 'request_id': 'req-7b4045fc-7125-462d-9214-63e546b536f0'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:57.050512 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e443575-5df0-45cd-9aed-ebf8c4f6f012 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:57.183929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-7b4045fc-7125-462d-9214-63e546b536f0 admin admin] The token cache is being invalidate because project 28a7633a75a745049c541513cd4b1120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:55:57.184766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-7b4045fc-7125-462d-9214-63e546b536f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:57.185630 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-7b4045fc-7125-462d-9214-63e546b536f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:55:57.190052 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 835/1701] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:55:56 2022] DELETE /identity/v3/projects/28a7633a75a745049c541513cd4b1120 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:57.191957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3799a-73fc-4c49-be98-5591732297f9 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:57.193588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3799a-73fc-4c49-be98-5591732297f9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:57.201517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3799a-73fc-4c49-be98-5591732297f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:57.206079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdf850-1eae-48dd-bdcd-a589156aabc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:57.206844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf850-1eae-48dd-bdcd-a589156aabc9 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:57.207054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf850-1eae-48dd-bdcd-a589156aabc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:57.207345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdf850-1eae-48dd-bdcd-a589156aabc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:57.207982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdf850-1eae-48dd-bdcd-a589156aabc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:57.209684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdf850-1eae-48dd-bdcd-a589156aabc9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:57.244623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 867/1702] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:57.244623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a15f8-012a-443b-b589-2b7f23acf23d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:57.244623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a15f8-012a-443b-b589-2b7f23acf23d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:57.244623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a15f8-012a-443b-b589-2b7f23acf23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:57.285615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b66a15f8-012a-443b-b589-2b7f23acf23d None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:57.286885 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b66a15f8-012a-443b-b589-2b7f23acf23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:57.300818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 868/1703] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:57.301708 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a15f8-012a-443b-b589-2b7f23acf23d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:57.308194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b66a15f8-012a-443b-b589-2b7f23acf23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:57.308363 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-fea3799a-73fc-4c49-be98-5591732297f9 None None] MFA Rules not processed for user `664cd13a536d49d387d96c53e40776f4`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:57.310060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-fea3799a-73fc-4c49-be98-5591732297f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:57.345817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a15f8-012a-443b-b589-2b7f23acf23d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:57.354492 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af06103-1d5d-49d8-a4d4-ec7650b101b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:57.355168 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af06103-1d5d-49d8-a4d4-ec7650b101b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:57.355458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af06103-1d5d-49d8-a4d4-ec7650b101b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:57.355728 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af06103-1d5d-49d8-a4d4-ec7650b101b7 admin admin] PATH_INFO: `/v3/projects/7bfaa4710edf4953a8301100191078b5/users/4146306a979048bcb9bdfe197a2058c1/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:57.388594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af06103-1d5d-49d8-a4d4-ec7650b101b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bfaa4710edf4953a8301100191078b5, user_id=4146306a979048bcb9bdfe197a2058c1, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:57.393343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af06103-1d5d-49d8-a4d4-ec7650b101b7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:57.422440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 836/1704] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:57 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:57.423988 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f00b1-7cda-46f4-a85b-4d5217adadb8 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:57.424277 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f00b1-7cda-46f4-a85b-4d5217adadb8 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:57.424927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f00b1-7cda-46f4-a85b-4d5217adadb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:57.460114 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b46f00b1-7cda-46f4-a85b-4d5217adadb8 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:57.461383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b46f00b1-7cda-46f4-a85b-4d5217adadb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:57.476390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 837/1705] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:57.476390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f00b1-7cda-46f4-a85b-4d5217adadb8 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:57.476390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b46f00b1-7cda-46f4-a85b-4d5217adadb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:57.507487 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 869/1706] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:57 2022] PUT /identity/v3/projects/7bfaa4710edf4953a8301100191078b5/users/4146306a979048bcb9bdfe197a2058c1/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:57.507487 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af06103-1d5d-49d8-a4d4-ec7650b101b7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:57.607496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f00b1-7cda-46f4-a85b-4d5217adadb8 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:57.616891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af06103-1d5d-49d8-a4d4-ec7650b101b7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:57.648890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198d04b-b47d-4583-81b3-5cf8aa2b9391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:57.649628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198d04b-b47d-4583-81b3-5cf8aa2b9391 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:57.649993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198d04b-b47d-4583-81b3-5cf8aa2b9391 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:57.650881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198d04b-b47d-4583-81b3-5cf8aa2b9391 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:57.652656 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198d04b-b47d-4583-81b3-5cf8aa2b9391 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:55:57.652980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198d04b-b47d-4583-81b3-5cf8aa2b9391 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:57.656065 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198d04b-b47d-4583-81b3-5cf8aa2b9391 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:57.670109 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 870/1707] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:55:57.671305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de152fb-51cd-431b-9b28-42a01cc6ab8a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:57.671594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de152fb-51cd-431b-9b28-42a01cc6ab8a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:57.671869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de152fb-51cd-431b-9b28-42a01cc6ab8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:57.692450 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d6069-27e8-490c-841e-0bafa76e9561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:57.692892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d6069-27e8-490c-841e-0bafa76e9561 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:57.692892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d6069-27e8-490c-841e-0bafa76e9561 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:57.693069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d6069-27e8-490c-841e-0bafa76e9561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:57.725479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7de152fb-51cd-431b-9b28-42a01cc6ab8a None None] MFA Rules not processed for user `4146306a979048bcb9bdfe197a2058c1`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:57.727588 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7de152fb-51cd-431b-9b28-42a01cc6ab8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:57.772491 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d6069-27e8-490c-841e-0bafa76e9561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:57.774295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d6069-27e8-490c-841e-0bafa76e9561 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:57.785076 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 838/1708] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:57 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:57.786941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cdd45-597d-49fd-92f2-182f014eb378 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:57.788250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cdd45-597d-49fd-92f2-182f014eb378 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:57.788653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7cdd45-597d-49fd-92f2-182f014eb378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:57.838665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 871/1709] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:57.839875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de152fb-51cd-431b-9b28-42a01cc6ab8a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:57.840150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-ca7cdd45-597d-49fd-92f2-182f014eb378 None None] MFA Rules not processed for user `2bf1c7a6cc2e4b1294e7b162a8aba738`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:57.843398 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ca7cdd45-597d-49fd-92f2-182f014eb378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:57.898190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de152fb-51cd-431b-9b28-42a01cc6ab8a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:57.926612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c0551-11fc-414b-b23d-10a71a61db5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:57.929308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c0551-11fc-414b-b23d-10a71a61db5d admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:57.929698 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c0551-11fc-414b-b23d-10a71a61db5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:57.929929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c0551-11fc-414b-b23d-10a71a61db5d admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:57.933535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c0551-11fc-414b-b23d-10a71a61db5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:57.939345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c0551-11fc-414b-b23d-10a71a61db5d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:58.009946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 839/1710] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:58.009946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732b357-db17-4b49-8b4b-84934c97aefc None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:58.009946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732b357-db17-4b49-8b4b-84934c97aefc None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:58.009946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732b357-db17-4b49-8b4b-84934c97aefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:58.095729 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 872/1711] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:55:57 2022] POST /identity/v3/projects => generated 403 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:58.096689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c0551-11fc-414b-b23d-10a71a61db5d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:58.101291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-799c0551-11fc-414b-b23d-10a71a61db5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:58.133066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b732b357-db17-4b49-8b4b-84934c97aefc None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:58.134348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b732b357-db17-4b49-8b4b-84934c97aefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:58.267048 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 840/1712] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 19:55:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:58.267991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732b357-db17-4b49-8b4b-84934c97aefc None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:58.298096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c0551-11fc-414b-b23d-10a71a61db5d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:58.318407 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46824df5-3f81-455d-bc66-5f5c64c4f5f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:58.319709 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46824df5-3f81-455d-bc66-5f5c64c4f5f9 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:58.320218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46824df5-3f81-455d-bc66-5f5c64c4f5f9 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:58.320683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46824df5-3f81-455d-bc66-5f5c64c4f5f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:58.373472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732b357-db17-4b49-8b4b-84934c97aefc None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:58.379927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83b59b-40b3-4993-a01f-15b24cfc049e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:58.381204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83b59b-40b3-4993-a01f-15b24cfc049e admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:58.381607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83b59b-40b3-4993-a01f-15b24cfc049e admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:58.382185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a83b59b-40b3-4993-a01f-15b24cfc049e admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:58.382421 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83b59b-40b3-4993-a01f-15b24cfc049e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:58.384419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a83b59b-40b3-4993-a01f-15b24cfc049e admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:58.416645 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46824df5-3f81-455d-bc66-5f5c64c4f5f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:58.418323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46824df5-3f81-455d-bc66-5f5c64c4f5f9 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:58.444296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 873/1713] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:58.449984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46824df5-3f81-455d-bc66-5f5c64c4f5f9 service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:58.543951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 841/1714] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:55:58 2022] POST /identity/v3/users => generated 302 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:55:58.544742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83b59b-40b3-4993-a01f-15b24cfc049e admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:58.581677 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a83b59b-40b3-4993-a01f-15b24cfc049e admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:58.599388 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16c1fc-7d78-45b2-90d4-21ec0c115d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:58.600972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16c1fc-7d78-45b2-90d4-21ec0c115d2a admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:58.601256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16c1fc-7d78-45b2-90d4-21ec0c115d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:58.601529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f16c1fc-7d78-45b2-90d4-21ec0c115d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:58.602405 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16c1fc-7d78-45b2-90d4-21ec0c115d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:58.607432 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f16c1fc-7d78-45b2-90d4-21ec0c115d2a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:58.622157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46824df5-3f81-455d-bc66-5f5c64c4f5f9 service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:58.626358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6504c-ff54-4c9a-9181-cb6a7bfced79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:58.627052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6504c-ff54-4c9a-9181-cb6a7bfced79 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:58.627461 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6504c-ff54-4c9a-9181-cb6a7bfced79 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:58.627851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6504c-ff54-4c9a-9181-cb6a7bfced79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:58.630359 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-09b6504c-ff54-4c9a-9181-cb6a7bfced79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:58.633784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 842/1715] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:58.640473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16c1fc-7d78-45b2-90d4-21ec0c115d2a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:58.664223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f16c1fc-7d78-45b2-90d4-21ec0c115d2a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:58.676872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c7333-ea40-4f7d-8115-6ba3c481ace7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:58.677716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c7333-ea40-4f7d-8115-6ba3c481ace7 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:58.678049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c7333-ea40-4f7d-8115-6ba3c481ace7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:58.678326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c7333-ea40-4f7d-8115-6ba3c481ace7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:58.678964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c7333-ea40-4f7d-8115-6ba3c481ace7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:58.681584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c7333-ea40-4f7d-8115-6ba3c481ace7 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:58.739954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 843/1716] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:55:58.747798 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:58.750175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c7333-ea40-4f7d-8115-6ba3c481ace7 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:58.765579 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6504c-ff54-4c9a-9181-cb6a7bfced79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:58.767698 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6504c-ff54-4c9a-9181-cb6a7bfced79 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:58.775069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c7333-ea40-4f7d-8115-6ba3c481ace7 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:58.780274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 874/1717] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:58 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:58.781396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6504c-ff54-4c9a-9181-cb6a7bfced79 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:58.783382 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-09b6504c-ff54-4c9a-9181-cb6a7bfced79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:58.803416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02959b42-09a4-47ff-9fb1-4d4537d840fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:58.804220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:55:58.804949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02959b42-09a4-47ff-9fb1-4d4537d840fd admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:58.806208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02959b42-09a4-47ff-9fb1-4d4537d840fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:58.806962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02959b42-09a4-47ff-9fb1-4d4537d840fd admin admin] PATH_INFO: `/v3/projects/64f709ba93664bcabd7267318a071765/users/a79c4edc8b9640f3b3e673e72728f601/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:58.894737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02959b42-09a4-47ff-9fb1-4d4537d840fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=64f709ba93664bcabd7267318a071765, user_id=a79c4edc8b9640f3b3e673e72728f601, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:58.900611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02959b42-09a4-47ff-9fb1-4d4537d840fd admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:59.088505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 844/1718] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:55:58 2022] PUT /identity/v3/projects/64f709ba93664bcabd7267318a071765/users/a79c4edc8b9640f3b3e673e72728f601/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:55:59.095501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e1777-3462-4be7-b71a-2d54ee081acb None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:59.096398 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e1777-3462-4be7-b71a-2d54ee081acb None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:59.096852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2e1777-3462-4be7-b71a-2d54ee081acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:59.102957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6504c-ff54-4c9a-9181-cb6a7bfced79 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:59.160406 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-3c2e1777-3462-4be7-b71a-2d54ee081acb None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:59.161922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-3c2e1777-3462-4be7-b71a-2d54ee081acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:59.221337 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69bfdec-b697-4247-b529-ebbcca29b138 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:59.223082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bfdec-b697-4247-b529-ebbcca29b138 service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:59.230938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bfdec-b697-4247-b529-ebbcca29b138 service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:59.230938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69bfdec-b697-4247-b529-ebbcca29b138 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:59.310443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 845/1719] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:55:59.340808 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e1777-3462-4be7-b71a-2d54ee081acb None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:59.342447 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-3c2e1777-3462-4be7-b71a-2d54ee081acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:59.479309 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bfdec-b697-4247-b529-ebbcca29b138 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:59.482865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69bfdec-b697-4247-b529-ebbcca29b138 service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:59.491410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2e1777-3462-4be7-b71a-2d54ee081acb None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:59.498956 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 875/1720] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 19:55:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 718 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 19:55:59.501174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b61c9cd-e7c0-40b1-891f-7ddf3abd8eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:59.501174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61c9cd-e7c0-40b1-891f-7ddf3abd8eef service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:59.501388 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61c9cd-e7c0-40b1-891f-7ddf3abd8eef service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:59.501652 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b61c9cd-e7c0-40b1-891f-7ddf3abd8eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:59.507792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a81a7-4dbe-4ef1-863d-53333ce79d6d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:59.508087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a81a7-4dbe-4ef1-863d-53333ce79d6d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:59.508390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a81a7-4dbe-4ef1-863d-53333ce79d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:59.511749 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61c9cd-e7c0-40b1-891f-7ddf3abd8eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:59.513537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b61c9cd-e7c0-40b1-891f-7ddf3abd8eef service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:59.528814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 846/1721] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:55:59.580419 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-348a81a7-4dbe-4ef1-863d-53333ce79d6d None None] MFA Rules not processed for user `268dbf54f29c40d9a4daa772c44735d4`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:59.597571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-348a81a7-4dbe-4ef1-863d-53333ce79d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:59.758696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d51fb3-1905-42d2-a183-7ddca2a2df09 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:59.759212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d51fb3-1905-42d2-a183-7ddca2a2df09 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:59.759769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d51fb3-1905-42d2-a183-7ddca2a2df09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:59.821755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b7d51fb3-1905-42d2-a183-7ddca2a2df09 None None] MFA Rules not processed for user `7103ff9544524393a8c7175998dcae3b`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:55:59.824298 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b7d51fb3-1905-42d2-a183-7ddca2a2df09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:55:59.844980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 876/1722] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:55:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:55:59.853896 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 847/1723] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Sep 14 19:55:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:55:59.899170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a81a7-4dbe-4ef1-863d-53333ce79d6d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:55:59.912982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a81a7-4dbe-4ef1-863d-53333ce79d6d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:55:59.931435 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323f609f-44d0-4220-b02b-34f73720a9c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:55:59.933163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f609f-44d0-4220-b02b-34f73720a9c7 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:55:59.938687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f609f-44d0-4220-b02b-34f73720a9c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:55:59.939132 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323f609f-44d0-4220-b02b-34f73720a9c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:55:59.966975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f609f-44d0-4220-b02b-34f73720a9c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:55:59.968624 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323f609f-44d0-4220-b02b-34f73720a9c7 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:55:59.996358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 877/1724] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:55:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:01.336934 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:02.301662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066b66f-d1cc-4d45-9001-b8990ebe177e None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:02.302375 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066b66f-d1cc-4d45-9001-b8990ebe177e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:02.302375 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066b66f-d1cc-4d45-9001-b8990ebe177e None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:02.303585 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 848/1725] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 19:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:56:02.362269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31199c31-a0f1-454f-93ef-a3f5e37dadd6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:02.363415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31199c31-a0f1-454f-93ef-a3f5e37dadd6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:02.363908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31199c31-a0f1-454f-93ef-a3f5e37dadd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:02.438451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-31199c31-a0f1-454f-93ef-a3f5e37dadd6 None None] MFA Rules not processed for user `2a4284b59f804dc59674d621d87634bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:02.442267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-31199c31-a0f1-454f-93ef-a3f5e37dadd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:02.464446 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:02.651727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 878/1726] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 19:56:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:03.744433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:03.812040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066b66f-d1cc-4d45-9001-b8990ebe177e None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:03.850878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066b66f-d1cc-4d45-9001-b8990ebe177e None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:03.864398 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cf3056-7783-40db-8a08-194ebdb20f02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:03.872426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf3056-7783-40db-8a08-194ebdb20f02 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:03.872426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf3056-7783-40db-8a08-194ebdb20f02 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:03.872426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cf3056-7783-40db-8a08-194ebdb20f02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:03.894371 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf3056-7783-40db-8a08-194ebdb20f02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:03.896345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cf3056-7783-40db-8a08-194ebdb20f02 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:03.919389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 849/1727] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:05.639099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf35768-67f7-4d34-9ea3-270283b58ef6 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:05.639099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf35768-67f7-4d34-9ea3-270283b58ef6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:05.639099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf35768-67f7-4d34-9ea3-270283b58ef6 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:05.640597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 879/1728] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 19:56:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:56:05.799703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4546-7190-4522-9da3-53c65319bdad None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:05.800743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4546-7190-4522-9da3-53c65319bdad None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:05.801030 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beff4546-7190-4522-9da3-53c65319bdad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:05.886174 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf35768-67f7-4d34-9ea3-270283b58ef6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:05.906229 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-beff4546-7190-4522-9da3-53c65319bdad None None] MFA Rules not processed for user `2a4284b59f804dc59674d621d87634bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:05.918097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-beff4546-7190-4522-9da3-53c65319bdad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:05.941356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf35768-67f7-4d34-9ea3-270283b58ef6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:05.979943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e610f-25ad-4b4d-b483-e88ec195fc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:05.980609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e610f-25ad-4b4d-b483-e88ec195fc5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:05.980890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e610f-25ad-4b4d-b483-e88ec195fc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:05.992139 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e610f-25ad-4b4d-b483-e88ec195fc5d admin admin] PATH_INFO: `/v3/users/4146306a979048bcb9bdfe197a2058c1` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:05.995081 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e610f-25ad-4b4d-b483-e88ec195fc5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4146306a979048bcb9bdfe197a2058c1)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:05.997082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e610f-25ad-4b4d-b483-e88ec195fc5d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:06.003845 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 850/1729] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 19:56:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:06.412911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-106e610f-25ad-4b4d-b483-e88ec195fc5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4146306a979048bcb9bdfe197a2058c1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:06.509975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-106e610f-25ad-4b4d-b483-e88ec195fc5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4146306a979048bcb9bdfe197a2058c1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:06.571553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-106e610f-25ad-4b4d-b483-e88ec195fc5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4146306a979048bcb9bdfe197a2058c1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:06.573117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-106e610f-25ad-4b4d-b483-e88ec195fc5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4146306a979048bcb9bdfe197a2058c1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:06.578865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-106e610f-25ad-4b4d-b483-e88ec195fc5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4146306a979048bcb9bdfe197a2058c1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:06.619721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 880/1730] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:05 2022] DELETE /identity/v3/users/4146306a979048bcb9bdfe197a2058c1 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:06.624926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ea33-ab78-4ded-98c5-744d11af534f None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:06.625240 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ea33-ab78-4ded-98c5-744d11af534f None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:06.625538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ea33-ab78-4ded-98c5-744d11af534f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:06.650902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b427ea33-ab78-4ded-98c5-744d11af534f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:06.652412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b427ea33-ab78-4ded-98c5-744d11af534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:06.698961 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:06.817329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b617c0-a780-44b2-b734-e25bd2c74023 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:06.817329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b617c0-a780-44b2-b734-e25bd2c74023 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:06.817329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b617c0-a780-44b2-b734-e25bd2c74023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:06.845895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 851/1731] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:06.847808 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427ea33-ab78-4ded-98c5-744d11af534f None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:06.851062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b427ea33-ab78-4ded-98c5-744d11af534f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:06.915852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-01b617c0-a780-44b2-b734-e25bd2c74023 None None] MFA Rules not processed for user `a79c4edc8b9640f3b3e673e72728f601`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:06.918866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-01b617c0-a780-44b2-b734-e25bd2c74023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:07.092079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427ea33-ab78-4ded-98c5-744d11af534f None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:07.102756 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 881/1732] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:06 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:07.106574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b617c0-a780-44b2-b734-e25bd2c74023 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:07.108910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-01b617c0-a780-44b2-b734-e25bd2c74023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:07.110546 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f41de-f0f7-48de-b22d-42c8e88657f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:07.111183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f41de-f0f7-48de-b22d-42c8e88657f0 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:07.111464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f41de-f0f7-48de-b22d-42c8e88657f0 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:07.111732 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f41de-f0f7-48de-b22d-42c8e88657f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:07.122305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-f18f41de-f0f7-48de-b22d-42c8e88657f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:07.166124 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f41de-f0f7-48de-b22d-42c8e88657f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:07.167825 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f41de-f0f7-48de-b22d-42c8e88657f0 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:07.197033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:07.197541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 852/1733] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:07.201006 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f41de-f0f7-48de-b22d-42c8e88657f0 service nova] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:07.211237 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-f18f41de-f0f7-48de-b22d-42c8e88657f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:07.247684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f41de-f0f7-48de-b22d-42c8e88657f0 service nova] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:07.261161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45d378-fb65-408e-a1fa-0b84cd2c488a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:07.261934 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45d378-fb65-408e-a1fa-0b84cd2c488a admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:07.262577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45d378-fb65-408e-a1fa-0b84cd2c488a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:07.263269 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45d378-fb65-408e-a1fa-0b84cd2c488a admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:07.278204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45d378-fb65-408e-a1fa-0b84cd2c488a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:07.283655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45d378-fb65-408e-a1fa-0b84cd2c488a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:07.303311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b617c0-a780-44b2-b734-e25bd2c74023 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:07.306944 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6e973-6e62-4a36-933d-bbc38e680545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:07.307692 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6e973-6e62-4a36-933d-bbc38e680545 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:07.308136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6e973-6e62-4a36-933d-bbc38e680545 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:07.308517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6e973-6e62-4a36-933d-bbc38e680545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:07.324680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6e973-6e62-4a36-933d-bbc38e680545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:07.329005 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6e973-6e62-4a36-933d-bbc38e680545 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:07.367338 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 882/1734] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:07.447122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 853/1735] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:07 2022] POST /identity/v3/projects => generated 403 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:07.451438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6e973-6e62-4a36-933d-bbc38e680545 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:07.465425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6e973-6e62-4a36-933d-bbc38e680545 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:07.630671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c18fc8-82a5-46b0-b28b-e395e8d0ee32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:07.631556 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c18fc8-82a5-46b0-b28b-e395e8d0ee32 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:07.632013 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c18fc8-82a5-46b0-b28b-e395e8d0ee32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:07.632446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c18fc8-82a5-46b0-b28b-e395e8d0ee32 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:07.633222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c18fc8-82a5-46b0-b28b-e395e8d0ee32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:07.636347 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c18fc8-82a5-46b0-b28b-e395e8d0ee32 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:07.692975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45d378-fb65-408e-a1fa-0b84cd2c488a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:07.695258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-7d45d378-fb65-408e-a1fa-0b84cd2c488a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:07.738382 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45d378-fb65-408e-a1fa-0b84cd2c488a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:07.752035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc4e6d-061a-4d01-a7fa-09de4be5fb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:07.752708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc4e6d-061a-4d01-a7fa-09de4be5fb54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:07.755609 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc4e6d-061a-4d01-a7fa-09de4be5fb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:07.755609 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc4e6d-061a-4d01-a7fa-09de4be5fb54 admin admin] PATH_INFO: `/v3/users/9b5a572ff16844a795f44f486cb3cb4e` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:07.755609 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc4e6d-061a-4d01-a7fa-09de4be5fb54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b5a572ff16844a795f44f486cb3cb4e)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:07.771381 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc4e6d-061a-4d01-a7fa-09de4be5fb54 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:07.789865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 883/1736] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:07 2022] POST /identity/v3/users => generated 308 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:07.794572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c18fc8-82a5-46b0-b28b-e395e8d0ee32 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:07.823634 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c18fc8-82a5-46b0-b28b-e395e8d0ee32 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:07.864040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f6ba8-5872-4190-9415-fbcd064b2a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:07.864040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f6ba8-5872-4190-9415-fbcd064b2a85 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:07.864040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f6ba8-5872-4190-9415-fbcd064b2a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:07.864040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f6ba8-5872-4190-9415-fbcd064b2a85 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:07.864334 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f6ba8-5872-4190-9415-fbcd064b2a85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:07.864334 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9f6ba8-5872-4190-9415-fbcd064b2a85 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:07.913076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 884/1737] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:07.914440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f6ba8-5872-4190-9415-fbcd064b2a85 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:07.925106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f6ba8-5872-4190-9415-fbcd064b2a85 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:07.930524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783127d-c309-4ec1-8033-46dde30e521c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:07.931451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783127d-c309-4ec1-8033-46dde30e521c admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:07.931912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783127d-c309-4ec1-8033-46dde30e521c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:07.932430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783127d-c309-4ec1-8033-46dde30e521c admin admin] PATH_INFO: `/v3/projects/58498d0c2ed14f99bf5b98e1a9045775/users/9b60c43c67a842d6bb40d285b40ed5d9/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:07.957971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783127d-c309-4ec1-8033-46dde30e521c admin admin] RBAC: Authorizing `identity:create_grant(project_id=58498d0c2ed14f99bf5b98e1a9045775, user_id=9b60c43c67a842d6bb40d285b40ed5d9, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:07.962705 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783127d-c309-4ec1-8033-46dde30e521c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:08.166153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 885/1738] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:07 2022] PUT /identity/v3/projects/58498d0c2ed14f99bf5b98e1a9045775/users/9b60c43c67a842d6bb40d285b40ed5d9/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:08.167367 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783127d-c309-4ec1-8033-46dde30e521c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:08.169815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-86bc4e6d-061a-4d01-a7fa-09de4be5fb54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b5a572ff16844a795f44f486cb3cb4e'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:08.195486 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-86bc4e6d-061a-4d01-a7fa-09de4be5fb54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b5a572ff16844a795f44f486cb3cb4e'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:08.287891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-86bc4e6d-061a-4d01-a7fa-09de4be5fb54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b5a572ff16844a795f44f486cb3cb4e'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:08.288460 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-86bc4e6d-061a-4d01-a7fa-09de4be5fb54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b5a572ff16844a795f44f486cb3cb4e'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:08.298197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-86bc4e6d-061a-4d01-a7fa-09de4be5fb54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b5a572ff16844a795f44f486cb3cb4e'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:08.300789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783127d-c309-4ec1-8033-46dde30e521c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:08.312768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a6617-bcb9-40ca-a4ba-78e52cc4153c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:08.314139 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a6617-bcb9-40ca-a4ba-78e52cc4153c admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:08.314703 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a6617-bcb9-40ca-a4ba-78e52cc4153c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:08.315257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a6617-bcb9-40ca-a4ba-78e52cc4153c admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:08.316209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a6617-bcb9-40ca-a4ba-78e52cc4153c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:08.320307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a6617-bcb9-40ca-a4ba-78e52cc4153c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:08.321588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 854/1739] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:07 2022] DELETE /identity/v3/users/9b5a572ff16844a795f44f486cb3cb4e => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:08.323146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d30ee-0cb1-401d-a392-c7c010fc5a17 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:08.326144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d30ee-0cb1-401d-a392-c7c010fc5a17 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:08.326144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d30ee-0cb1-401d-a392-c7c010fc5a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:08.339509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 886/1740] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:08.346168 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a6617-bcb9-40ca-a4ba-78e52cc4153c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:08.349062 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-012a6617-bcb9-40ca-a4ba-78e52cc4153c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:08.370761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-852d30ee-0cb1-401d-a392-c7c010fc5a17 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:08.372085 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-852d30ee-0cb1-401d-a392-c7c010fc5a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:08.389580 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a6617-bcb9-40ca-a4ba-78e52cc4153c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:08.535888 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:08.553947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 855/1741] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:08.566193 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d30ee-0cb1-401d-a392-c7c010fc5a17 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:08.571837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-852d30ee-0cb1-401d-a392-c7c010fc5a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:08.587440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda47f1a-fc97-4fe5-8bfd-1166c6e42995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:08.591939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda47f1a-fc97-4fe5-8bfd-1166c6e42995 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:08.591939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda47f1a-fc97-4fe5-8bfd-1166c6e42995 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:08.591939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda47f1a-fc97-4fe5-8bfd-1166c6e42995 admin admin] PATH_INFO: `/v3/projects/58498d0c2ed14f99bf5b98e1a9045775/users/9b60c43c67a842d6bb40d285b40ed5d9/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:08.601187 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda47f1a-fc97-4fe5-8bfd-1166c6e42995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58498d0c2ed14f99bf5b98e1a9045775, user_id=9b60c43c67a842d6bb40d285b40ed5d9, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:08.607892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda47f1a-fc97-4fe5-8bfd-1166c6e42995 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:08.634311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 887/1742] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:08 2022] PUT /identity/v3/projects/58498d0c2ed14f99bf5b98e1a9045775/users/9b60c43c67a842d6bb40d285b40ed5d9/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:08.708851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d30ee-0cb1-401d-a392-c7c010fc5a17 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:08.800244 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c6f06c-cbdd-47a2-9d29-50c92f410a6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:08.807436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6f06c-cbdd-47a2-9d29-50c92f410a6c service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:08.809003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6f06c-cbdd-47a2-9d29-50c92f410a6c service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:08.809362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c6f06c-cbdd-47a2-9d29-50c92f410a6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:09.005046 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6f06c-cbdd-47a2-9d29-50c92f410a6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:09.012281 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c6f06c-cbdd-47a2-9d29-50c92f410a6c service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:09.031295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 856/1743] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:09.327368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda47f1a-fc97-4fe5-8bfd-1166c6e42995 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:09.332444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-dda47f1a-fc97-4fe5-8bfd-1166c6e42995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:09.351509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda47f1a-fc97-4fe5-8bfd-1166c6e42995 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:09.369859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b905d-fef7-4624-be2f-3be59e280008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:09.370623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b905d-fef7-4624-be2f-3be59e280008 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:09.372738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b905d-fef7-4624-be2f-3be59e280008 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:09.372738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b905d-fef7-4624-be2f-3be59e280008 admin admin] PATH_INFO: `/v3/projects/7bfaa4710edf4953a8301100191078b5` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:09.372738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b905d-fef7-4624-be2f-3be59e280008 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bfaa4710edf4953a8301100191078b5)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:09.395122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b905d-fef7-4624-be2f-3be59e280008 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:09.610974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-226b905d-fef7-4624-be2f-3be59e280008 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bfaa4710edf4953a8301100191078b5', 'request_id': 'req-226b905d-fef7-4624-be2f-3be59e280008'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:09.738820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-226b905d-fef7-4624-be2f-3be59e280008 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bfaa4710edf4953a8301100191078b5', 'request_id': 'req-226b905d-fef7-4624-be2f-3be59e280008'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:09.760480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-226b905d-fef7-4624-be2f-3be59e280008 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bfaa4710edf4953a8301100191078b5', 'request_id': 'req-226b905d-fef7-4624-be2f-3be59e280008'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:09.789048 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-226b905d-fef7-4624-be2f-3be59e280008 admin admin] The token cache is being invalidate because project 7bfaa4710edf4953a8301100191078b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:09.789667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-226b905d-fef7-4624-be2f-3be59e280008 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:09.793961 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-226b905d-fef7-4624-be2f-3be59e280008 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:09.796439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 888/1744] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:09 2022] DELETE /identity/v3/projects/7bfaa4710edf4953a8301100191078b5 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:09.802439 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee6466-5525-4063-91e6-60c6ccdf0e8a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:09.803628 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee6466-5525-4063-91e6-60c6ccdf0e8a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:09.804078 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee6466-5525-4063-91e6-60c6ccdf0e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:09.839865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-58ee6466-5525-4063-91e6-60c6ccdf0e8a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:09.841528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-58ee6466-5525-4063-91e6-60c6ccdf0e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:09.876737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:09.964477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 857/1745] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:09.973164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662e217-a422-426a-9e3b-7bac3d800bdf None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:09.973473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662e217-a422-426a-9e3b-7bac3d800bdf None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:09.973721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662e217-a422-426a-9e3b-7bac3d800bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:10.026491 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3662e217-a422-426a-9e3b-7bac3d800bdf None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:10.028988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3662e217-a422-426a-9e3b-7bac3d800bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:10.057216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 889/1746] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:10.058256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee6466-5525-4063-91e6-60c6ccdf0e8a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:10.086257 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee6466-5525-4063-91e6-60c6ccdf0e8a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:10.104884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5330e-2685-4c0d-bdb5-a996d7d92262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:10.105529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5330e-2685-4c0d-bdb5-a996d7d92262 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:10.105828 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5330e-2685-4c0d-bdb5-a996d7d92262 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:10.106180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba5330e-2685-4c0d-bdb5-a996d7d92262 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:10.106856 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5330e-2685-4c0d-bdb5-a996d7d92262 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:10.107120 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5330e-2685-4c0d-bdb5-a996d7d92262 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:10.109971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba5330e-2685-4c0d-bdb5-a996d7d92262 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:10.172509 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 858/1747] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:56:10.177167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662e217-a422-426a-9e3b-7bac3d800bdf None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:10.188322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662e217-a422-426a-9e3b-7bac3d800bdf None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:10.198912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b206f-0c82-4d48-bacc-ae56cd653b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:10.199688 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b206f-0c82-4d48-bacc-ae56cd653b14 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:10.199799 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b206f-0c82-4d48-bacc-ae56cd653b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:10.200062 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b206f-0c82-4d48-bacc-ae56cd653b14 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:10.200689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b206f-0c82-4d48-bacc-ae56cd653b14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:10.214074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b206f-0c82-4d48-bacc-ae56cd653b14 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:10.392123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 890/1748] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:10 2022] POST /identity/v3/projects => generated 375 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:10.396240 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5330e-2685-4c0d-bdb5-a996d7d92262 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:10.409347 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba5330e-2685-4c0d-bdb5-a996d7d92262 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:10.558295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c152a45-803e-404a-97fc-94009459fde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:10.559100 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c152a45-803e-404a-97fc-94009459fde0 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:10.559610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c152a45-803e-404a-97fc-94009459fde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:10.559885 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c152a45-803e-404a-97fc-94009459fde0 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:10.560531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c152a45-803e-404a-97fc-94009459fde0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:10.563352 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c152a45-803e-404a-97fc-94009459fde0 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:10.718693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 859/1749] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:10 2022] POST /identity/v3/users => generated 288 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:10.723556 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b206f-0c82-4d48-bacc-ae56cd653b14 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:10.742292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b206f-0c82-4d48-bacc-ae56cd653b14 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:10.757577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e183ad-c3a1-405e-9501-c198c829ec5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:10.758243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e183ad-c3a1-405e-9501-c198c829ec5d admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:10.758525 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e183ad-c3a1-405e-9501-c198c829ec5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:10.761985 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e183ad-c3a1-405e-9501-c198c829ec5d admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:10.767938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e183ad-c3a1-405e-9501-c198c829ec5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:10.767938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e183ad-c3a1-405e-9501-c198c829ec5d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:10.814047 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 891/1750] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:10.814475 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c152a45-803e-404a-97fc-94009459fde0 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:10.836391 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c152a45-803e-404a-97fc-94009459fde0 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:10.847799 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd63e25-2e51-46f3-9c02-cc658d10c782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:10.849449 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd63e25-2e51-46f3-9c02-cc658d10c782 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:10.851854 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd63e25-2e51-46f3-9c02-cc658d10c782 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:10.853165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd63e25-2e51-46f3-9c02-cc658d10c782 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:10.863676 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd63e25-2e51-46f3-9c02-cc658d10c782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:10.866893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd63e25-2e51-46f3-9c02-cc658d10c782 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:10.889521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 860/1751] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:10.894927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e183ad-c3a1-405e-9501-c198c829ec5d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:10.917073 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e183ad-c3a1-405e-9501-c198c829ec5d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:10.924075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa8d21-6ef1-4560-b287-8c8e350bc56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:10.924977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa8d21-6ef1-4560-b287-8c8e350bc56f admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:10.925259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa8d21-6ef1-4560-b287-8c8e350bc56f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:10.930542 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efa8d21-6ef1-4560-b287-8c8e350bc56f admin admin] PATH_INFO: `/v3/projects/2a8e905c078c4edb89146bf44e7889e1/users/26130a0a358749ee903c6e1b46fe5862/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:10.954344 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa8d21-6ef1-4560-b287-8c8e350bc56f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a8e905c078c4edb89146bf44e7889e1, user_id=26130a0a358749ee903c6e1b46fe5862, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:10.957562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efa8d21-6ef1-4560-b287-8c8e350bc56f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:11.060472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 892/1752] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:10 2022] PUT /identity/v3/projects/2a8e905c078c4edb89146bf44e7889e1/users/26130a0a358749ee903c6e1b46fe5862/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:11.071162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f27340-5741-45d5-b657-925ff77c56c9 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:11.071446 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f27340-5741-45d5-b657-925ff77c56c9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:11.071874 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f27340-5741-45d5-b657-925ff77c56c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:11.125268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-d4f27340-5741-45d5-b657-925ff77c56c9 None None] MFA Rules not processed for user `26130a0a358749ee903c6e1b46fe5862`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:11.127356 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d4f27340-5741-45d5-b657-925ff77c56c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:11.281599 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 861/1753] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:11.300234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa8d21-6ef1-4560-b287-8c8e350bc56f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:11.300234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6efa8d21-6ef1-4560-b287-8c8e350bc56f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:11.305159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f27340-5741-45d5-b657-925ff77c56c9 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:11.308709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d4f27340-5741-45d5-b657-925ff77c56c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:11.367076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:11.501703 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efa8d21-6ef1-4560-b287-8c8e350bc56f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:11.509112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa262335-7707-416b-9eaa-c21253151e99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:11.509791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa262335-7707-416b-9eaa-c21253151e99 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:11.510122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa262335-7707-416b-9eaa-c21253151e99 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:11.510394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa262335-7707-416b-9eaa-c21253151e99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:11.519872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa262335-7707-416b-9eaa-c21253151e99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:11.521530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa262335-7707-416b-9eaa-c21253151e99 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:11.534758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 893/1754] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:11.633841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:11.641037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f27340-5741-45d5-b657-925ff77c56c9 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:11.656078 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:11.657516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:11.658459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:11.658459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] PATH_INFO: `/v3/projects/e0f3036a64644244a3b261c9ec408170` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:11.659731 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f3036a64644244a3b261c9ec408170)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:11.661743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:12.458220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa262335-7707-416b-9eaa-c21253151e99 service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:12.468582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa262335-7707-416b-9eaa-c21253151e99 service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:12.474685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216564eb-9812-4c62-8ea0-698133fe78f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:12.476170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216564eb-9812-4c62-8ea0-698133fe78f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:12.477680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216564eb-9812-4c62-8ea0-698133fe78f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:12.477680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216564eb-9812-4c62-8ea0-698133fe78f4 admin admin] PATH_INFO: `/v3/users/26130a0a358749ee903c6e1b46fe5862` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:12.478445 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216564eb-9812-4c62-8ea0-698133fe78f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26130a0a358749ee903c6e1b46fe5862)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:12.481406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216564eb-9812-4c62-8ea0-698133fe78f4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:12.698649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f3036a64644244a3b261c9ec408170', 'request_id': 'req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:12.698649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f3036a64644244a3b261c9ec408170', 'request_id': 'req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:12.699515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f3036a64644244a3b261c9ec408170', 'request_id': 'req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:13.200244 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] The token cache is being invalidate because project e0f3036a64644244a3b261c9ec408170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:13.200244 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:13.206223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:13.207915 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 862/1755] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:11 2022] DELETE /identity/v3/projects/e0f3036a64644244a3b261c9ec408170 => generated 0 bytes in 1903 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:13.211820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:13.213404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:13.309232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee263bce-ed52-49ae-bb66-93db1a0cc8d6 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:13.318359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597064c0-604f-434f-a308-587e39a3a06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:13.320784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597064c0-604f-434f-a308-587e39a3a06c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:13.321333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597064c0-604f-434f-a308-587e39a3a06c admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:13.322105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597064c0-604f-434f-a308-587e39a3a06c admin admin] PATH_INFO: `/v3/users/664cd13a536d49d387d96c53e40776f4` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:13.328723 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597064c0-604f-434f-a308-587e39a3a06c admin admin] RBAC: Authorizing `identity:delete_user(user_id=664cd13a536d49d387d96c53e40776f4)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:13.328723 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597064c0-604f-434f-a308-587e39a3a06c admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:13.336736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-216564eb-9812-4c62-8ea0-698133fe78f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26130a0a358749ee903c6e1b46fe5862'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:13.430256 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-216564eb-9812-4c62-8ea0-698133fe78f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26130a0a358749ee903c6e1b46fe5862'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:13.454045 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-216564eb-9812-4c62-8ea0-698133fe78f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26130a0a358749ee903c6e1b46fe5862'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:13.454796 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-216564eb-9812-4c62-8ea0-698133fe78f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26130a0a358749ee903c6e1b46fe5862'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:13.457231 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-216564eb-9812-4c62-8ea0-698133fe78f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26130a0a358749ee903c6e1b46fe5862'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:13.472340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 894/1756] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:12 2022] DELETE /identity/v3/users/26130a0a358749ee903c6e1b46fe5862 => generated 0 bytes in 1015 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:13.475168 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936ac0d-1b74-405a-b6ed-e70a0d6bd600 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:13.475168 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936ac0d-1b74-405a-b6ed-e70a0d6bd600 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:13.475168 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936ac0d-1b74-405a-b6ed-e70a0d6bd600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:13.509582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b936ac0d-1b74-405a-b6ed-e70a0d6bd600 None None] MFA Rules not processed for user `9b60c43c67a842d6bb40d285b40ed5d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:13.511035 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b936ac0d-1b74-405a-b6ed-e70a0d6bd600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:13.600606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 895/1757] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:13 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:13.602002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b23dc9-9cd3-4e10-834f-390881ec96a0 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:13.602358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b23dc9-9cd3-4e10-834f-390881ec96a0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:13.602696 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b23dc9-9cd3-4e10-834f-390881ec96a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:13.625374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-40b23dc9-9cd3-4e10-834f-390881ec96a0 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:13.628449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-40b23dc9-9cd3-4e10-834f-390881ec96a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:13.687359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-597064c0-604f-434f-a308-587e39a3a06c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '664cd13a536d49d387d96c53e40776f4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:13.705544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-597064c0-604f-434f-a308-587e39a3a06c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '664cd13a536d49d387d96c53e40776f4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:13.713420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 896/1758] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:13.728572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b23dc9-9cd3-4e10-834f-390881ec96a0 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:13.731138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-40b23dc9-9cd3-4e10-834f-390881ec96a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:13.736604 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:13.783848 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-597064c0-604f-434f-a308-587e39a3a06c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '664cd13a536d49d387d96c53e40776f4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:13.784479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-597064c0-604f-434f-a308-587e39a3a06c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '664cd13a536d49d387d96c53e40776f4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:13.794258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-597064c0-604f-434f-a308-587e39a3a06c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '664cd13a536d49d387d96c53e40776f4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:13.807609 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 863/1759] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:13 2022] DELETE /identity/v3/users/664cd13a536d49d387d96c53e40776f4 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:13.832524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b23dc9-9cd3-4e10-834f-390881ec96a0 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:13.894265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108807a-57bd-421f-99ce-3d795a96394e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:13.895015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108807a-57bd-421f-99ce-3d795a96394e service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:13.895227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108807a-57bd-421f-99ce-3d795a96394e service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:13.895795 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108807a-57bd-421f-99ce-3d795a96394e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:13.897845 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d108807a-57bd-421f-99ce-3d795a96394e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:13.959102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:14.018467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108807a-57bd-421f-99ce-3d795a96394e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:14.020688 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108807a-57bd-421f-99ce-3d795a96394e service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:14.033675 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 897/1760] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:56:14.633408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597064c0-604f-434f-a308-587e39a3a06c admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:14.635542 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-597064c0-604f-434f-a308-587e39a3a06c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:14.643203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987fd74-ec07-414d-b5e8-1b8eb6c5eb0a None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:14.643705 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987fd74-ec07-414d-b5e8-1b8eb6c5eb0a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:14.643802 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987fd74-ec07-414d-b5e8-1b8eb6c5eb0a None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:14.645582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 898/1761] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 19:56:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:56:14.654819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597064c0-604f-434f-a308-587e39a3a06c admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:14.662572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e4ea0-1dd9-4885-acba-f6fd7d802a86 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:14.663529 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e4ea0-1dd9-4885-acba-f6fd7d802a86 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:14.663529 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e4ea0-1dd9-4885-acba-f6fd7d802a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:14.666776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a84758-dd5f-4250-a665-002cac32a19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:14.669385 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a84758-dd5f-4250-a665-002cac32a19b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:14.669664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a84758-dd5f-4250-a665-002cac32a19b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:14.669969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a84758-dd5f-4250-a665-002cac32a19b admin admin] PATH_INFO: `/v3/projects/2a8e905c078c4edb89146bf44e7889e1` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:14.672216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a84758-dd5f-4250-a665-002cac32a19b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a8e905c078c4edb89146bf44e7889e1)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:14.674833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a84758-dd5f-4250-a665-002cac32a19b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:14.698952 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-651e4ea0-1dd9-4885-acba-f6fd7d802a86 None None] MFA Rules not processed for user `03bc280d0395479981b7c7b08c96001b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:14.701738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-651e4ea0-1dd9-4885-acba-f6fd7d802a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:14.778365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 899/1762] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 19:56:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:14.789535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e4ea0-1dd9-4885-acba-f6fd7d802a86 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:14.790959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-651e4ea0-1dd9-4885-acba-f6fd7d802a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:14.804273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e4ea0-1dd9-4885-acba-f6fd7d802a86 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:14.809767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594272b7-13a2-4cd4-9eef-a686084f4bbd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03bc280d0395479981b7c7b08c96001b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:14.810392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594272b7-13a2-4cd4-9eef-a686084f4bbd service placement] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:14.810666 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594272b7-13a2-4cd4-9eef-a686084f4bbd service placement] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:14.810934 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594272b7-13a2-4cd4-9eef-a686084f4bbd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:14.821152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594272b7-13a2-4cd4-9eef-a686084f4bbd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:14.822817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594272b7-13a2-4cd4-9eef-a686084f4bbd service placement] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:14.834066 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 900/1763] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:56:14.849292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594272b7-13a2-4cd4-9eef-a686084f4bbd service placement] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:14.851054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-594272b7-13a2-4cd4-9eef-a686084f4bbd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:14.884766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594272b7-13a2-4cd4-9eef-a686084f4bbd service placement] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:14.892037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-91a84758-dd5f-4250-a665-002cac32a19b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a8e905c078c4edb89146bf44e7889e1', 'request_id': 'req-91a84758-dd5f-4250-a665-002cac32a19b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:14.910514 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-91a84758-dd5f-4250-a665-002cac32a19b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a8e905c078c4edb89146bf44e7889e1', 'request_id': 'req-91a84758-dd5f-4250-a665-002cac32a19b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:14.923798 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-91a84758-dd5f-4250-a665-002cac32a19b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a8e905c078c4edb89146bf44e7889e1', 'request_id': 'req-91a84758-dd5f-4250-a665-002cac32a19b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:14.974303 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c540a2-bf8c-4072-a55f-bf3f0ff83925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:14.974942 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c540a2-bf8c-4072-a55f-bf3f0ff83925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:14.975217 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c540a2-bf8c-4072-a55f-bf3f0ff83925 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:14.975531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c540a2-bf8c-4072-a55f-bf3f0ff83925 admin admin] PATH_INFO: `/v3/projects/32193b443b1443de8699120f5f564840` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:14.976811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c540a2-bf8c-4072-a55f-bf3f0ff83925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32193b443b1443de8699120f5f564840)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:14.978915 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c540a2-bf8c-4072-a55f-bf3f0ff83925 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:15.007578 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-91a84758-dd5f-4250-a665-002cac32a19b admin admin] The token cache is being invalidate because project 2a8e905c078c4edb89146bf44e7889e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:15.013938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-91a84758-dd5f-4250-a665-002cac32a19b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:15.013938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-91a84758-dd5f-4250-a665-002cac32a19b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:15.013938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 864/1764] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:14 2022] DELETE /identity/v3/projects/2a8e905c078c4edb89146bf44e7889e1 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:15.016006 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3849a78-666b-45b5-a3b1-c10bb7819c6f None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.016347 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3849a78-666b-45b5-a3b1-c10bb7819c6f None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.017206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3849a78-666b-45b5-a3b1-c10bb7819c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.037513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-f3849a78-666b-45b5-a3b1-c10bb7819c6f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:15.046403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-f3849a78-666b-45b5-a3b1-c10bb7819c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:15.046805 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 865/1765] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:15.056259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ade26-ea24-45ca-a1aa-7370550b5fe3 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.056538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ade26-ea24-45ca-a1aa-7370550b5fe3 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.056857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4ade26-ea24-45ca-a1aa-7370550b5fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.081675 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-9b4ade26-ea24-45ca-a1aa-7370550b5fe3 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:15.081675 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-9b4ade26-ea24-45ca-a1aa-7370550b5fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:15.086283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 866/1766] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:15.090310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ade26-ea24-45ca-a1aa-7370550b5fe3 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:15.099333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4ade26-ea24-45ca-a1aa-7370550b5fe3 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:15.104166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b694f3-4391-4f90-85ac-b34ca126ed7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:15.105059 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b694f3-4391-4f90-85ac-b34ca126ed7b admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.105059 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b694f3-4391-4f90-85ac-b34ca126ed7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.105490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b694f3-4391-4f90-85ac-b34ca126ed7b admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.106812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b694f3-4391-4f90-85ac-b34ca126ed7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:15.107407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b694f3-4391-4f90-85ac-b34ca126ed7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:15.110941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b694f3-4391-4f90-85ac-b34ca126ed7b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:15.123469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 867/1767] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:56:15.128081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b694f3-4391-4f90-85ac-b34ca126ed7b admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:15.135178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-82c540a2-bf8c-4072-a55f-bf3f0ff83925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32193b443b1443de8699120f5f564840', 'request_id': 'req-82c540a2-bf8c-4072-a55f-bf3f0ff83925'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:15.144321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b694f3-4391-4f90-85ac-b34ca126ed7b admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:15.200435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b76c6-73ec-4973-a773-7253fce6ccc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:15.201056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b76c6-73ec-4973-a773-7253fce6ccc6 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.201330 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b76c6-73ec-4973-a773-7253fce6ccc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.201598 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b76c6-73ec-4973-a773-7253fce6ccc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.202240 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b76c6-73ec-4973-a773-7253fce6ccc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:15.204227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b76c6-73ec-4973-a773-7253fce6ccc6 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:15.233646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-82c540a2-bf8c-4072-a55f-bf3f0ff83925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32193b443b1443de8699120f5f564840', 'request_id': 'req-82c540a2-bf8c-4072-a55f-bf3f0ff83925'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:15.241546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-82c540a2-bf8c-4072-a55f-bf3f0ff83925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32193b443b1443de8699120f5f564840', 'request_id': 'req-82c540a2-bf8c-4072-a55f-bf3f0ff83925'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:15.252890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-82c540a2-bf8c-4072-a55f-bf3f0ff83925 admin admin] The token cache is being invalidate because project 32193b443b1443de8699120f5f564840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:15.253113 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-82c540a2-bf8c-4072-a55f-bf3f0ff83925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:15.253767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-82c540a2-bf8c-4072-a55f-bf3f0ff83925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:15.259044 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 901/1768] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:14 2022] DELETE /identity/v3/projects/32193b443b1443de8699120f5f564840 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:15.265868 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef919-8855-48db-83c8-2734d48b7181 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.265868 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef919-8855-48db-83c8-2734d48b7181 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.265868 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaef919-8855-48db-83c8-2734d48b7181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.281104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-9aaef919-8855-48db-83c8-2734d48b7181 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:15.282371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9aaef919-8855-48db-83c8-2734d48b7181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:15.290665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 902/1769] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:15.303490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f88625-ff1e-4012-959e-ab028c50ad8a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.303769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f88625-ff1e-4012-959e-ab028c50ad8a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.304079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f88625-ff1e-4012-959e-ab028c50ad8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.324972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-23f88625-ff1e-4012-959e-ab028c50ad8a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:15.328884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-23f88625-ff1e-4012-959e-ab028c50ad8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:15.333682 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 868/1770] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:15 2022] POST /identity/v3/projects => generated 389 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:56:15.337857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 903/1771] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:15.337990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b76c6-73ec-4973-a773-7253fce6ccc6 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:15.339037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f88625-ff1e-4012-959e-ab028c50ad8a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:15.340837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-337b76c6-73ec-4973-a773-7253fce6ccc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:15.349568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f88625-ff1e-4012-959e-ab028c50ad8a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:15.423249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b76c6-73ec-4973-a773-7253fce6ccc6 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:15.425261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f83b4c-0bfa-45ec-a688-ef6e2c5dea87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:15.426110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f83b4c-0bfa-45ec-a688-ef6e2c5dea87 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.426388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f83b4c-0bfa-45ec-a688-ef6e2c5dea87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.426689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f83b4c-0bfa-45ec-a688-ef6e2c5dea87 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.427575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f83b4c-0bfa-45ec-a688-ef6e2c5dea87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:15.427976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f83b4c-0bfa-45ec-a688-ef6e2c5dea87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:15.429648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f83b4c-0bfa-45ec-a688-ef6e2c5dea87 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:15.429770 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc994d3-a96c-460a-9feb-158350d5a73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:15.430390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc994d3-a96c-460a-9feb-158350d5a73b admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.431220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc994d3-a96c-460a-9feb-158350d5a73b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.431498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc994d3-a96c-460a-9feb-158350d5a73b admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.432390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc994d3-a96c-460a-9feb-158350d5a73b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:15.434837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc994d3-a96c-460a-9feb-158350d5a73b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:15.444225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 904/1772] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:56:15.448676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f83b4c-0bfa-45ec-a688-ef6e2c5dea87 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:15.459335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f83b4c-0bfa-45ec-a688-ef6e2c5dea87 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:15.464366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67820bb1-1b6d-4c03-9f07-201a51d4bc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:15.465017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67820bb1-1b6d-4c03-9f07-201a51d4bc54 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.465354 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67820bb1-1b6d-4c03-9f07-201a51d4bc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.465630 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67820bb1-1b6d-4c03-9f07-201a51d4bc54 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.466286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67820bb1-1b6d-4c03-9f07-201a51d4bc54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:15.468439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67820bb1-1b6d-4c03-9f07-201a51d4bc54 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:15.522081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 869/1773] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:15 2022] POST /identity/v3/users => generated 295 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:15.526347 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc994d3-a96c-460a-9feb-158350d5a73b admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:15.536638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc994d3-a96c-460a-9feb-158350d5a73b admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:15.541346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6437c-a597-4272-af12-e7b0ca5d364b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:15.541944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6437c-a597-4272-af12-e7b0ca5d364b admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.542223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6437c-a597-4272-af12-e7b0ca5d364b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.542493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a6437c-a597-4272-af12-e7b0ca5d364b admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.543155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6437c-a597-4272-af12-e7b0ca5d364b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:15.544747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a6437c-a597-4272-af12-e7b0ca5d364b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:15.560262 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 870/1774] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:56:15.562142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 905/1775] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:15 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:56:15.564917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6437c-a597-4272-af12-e7b0ca5d364b admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:15.566283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67820bb1-1b6d-4c03-9f07-201a51d4bc54 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:15.576177 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a6437c-a597-4272-af12-e7b0ca5d364b admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:15.662018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67820bb1-1b6d-4c03-9f07-201a51d4bc54 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:15.663292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf7b24-71d8-41e0-8e6c-2c59bc371ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:15.663954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf7b24-71d8-41e0-8e6c-2c59bc371ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.664241 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf7b24-71d8-41e0-8e6c-2c59bc371ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.664517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf7b24-71d8-41e0-8e6c-2c59bc371ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.665172 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf7b24-71d8-41e0-8e6c-2c59bc371ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:15.667073 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47af20b-ea8f-4356-8a74-35f0cccdf4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:15.667416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf7b24-71d8-41e0-8e6c-2c59bc371ef6 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:15.667682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47af20b-ea8f-4356-8a74-35f0cccdf4d6 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.667957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47af20b-ea8f-4356-8a74-35f0cccdf4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.668227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47af20b-ea8f-4356-8a74-35f0cccdf4d6 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.668851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47af20b-ea8f-4356-8a74-35f0cccdf4d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:15.670826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47af20b-ea8f-4356-8a74-35f0cccdf4d6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:15.680281 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 871/1776] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:15.684338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf7b24-71d8-41e0-8e6c-2c59bc371ef6 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:15.694613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf7b24-71d8-41e0-8e6c-2c59bc371ef6 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:15.699177 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cda17-41e1-46f3-a612-480b402407fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:15.700338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cda17-41e1-46f3-a612-480b402407fe admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.700338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cda17-41e1-46f3-a612-480b402407fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.700338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654cda17-41e1-46f3-a612-480b402407fe admin admin] PATH_INFO: `/v3/projects/0ef6c0779da24fd6af98228402011cfb/users/bc5f987003204c99929ab763ecbb6a0e/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.715747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cda17-41e1-46f3-a612-480b402407fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef6c0779da24fd6af98228402011cfb, user_id=bc5f987003204c99929ab763ecbb6a0e, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:15.722791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654cda17-41e1-46f3-a612-480b402407fe admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:15.836217 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 906/1777] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:15 2022] POST /identity/v3/users => generated 292 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:15.837248 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47af20b-ea8f-4356-8a74-35f0cccdf4d6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:15.838967 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c47af20b-ea8f-4356-8a74-35f0cccdf4d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:15.851178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47af20b-ea8f-4356-8a74-35f0cccdf4d6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:15.861977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88d0d8-6c3b-4773-b94e-a653d199d070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:15.862653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d0d8-6c3b-4773-b94e-a653d199d070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.862929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d0d8-6c3b-4773-b94e-a653d199d070 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.863266 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88d0d8-6c3b-4773-b94e-a653d199d070 admin admin] PATH_INFO: `/v3/users/a79c4edc8b9640f3b3e673e72728f601` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.864303 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88d0d8-6c3b-4773-b94e-a653d199d070 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a79c4edc8b9640f3b3e673e72728f601)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:15.866342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88d0d8-6c3b-4773-b94e-a653d199d070 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:15.885384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 872/1778] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:15 2022] PUT /identity/v3/projects/0ef6c0779da24fd6af98228402011cfb/users/bc5f987003204c99929ab763ecbb6a0e/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:15.886350 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cda17-41e1-46f3-a612-480b402407fe admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:15.960267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654cda17-41e1-46f3-a612-480b402407fe admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:15.965010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da31fcae-967a-4d0a-9863-b2354c121b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:15.965705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31fcae-967a-4d0a-9863-b2354c121b50 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.965978 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31fcae-967a-4d0a-9863-b2354c121b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.966252 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da31fcae-967a-4d0a-9863-b2354c121b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:15.966882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31fcae-967a-4d0a-9863-b2354c121b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:15.968509 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da31fcae-967a-4d0a-9863-b2354c121b50 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:15.981274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 873/1779] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:15.981274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569daf98-b6c7-4fa5-bbcb-d9f93fee825f None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:15.981274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569daf98-b6c7-4fa5-bbcb-d9f93fee825f None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:15.981274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569daf98-b6c7-4fa5-bbcb-d9f93fee825f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:16.008083 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-569daf98-b6c7-4fa5-bbcb-d9f93fee825f None None] MFA Rules not processed for user `bc5f987003204c99929ab763ecbb6a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:16.009326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-569daf98-b6c7-4fa5-bbcb-d9f93fee825f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:16.075147 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 874/1780] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:16.075992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569daf98-b6c7-4fa5-bbcb-d9f93fee825f None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:16.083546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3c88d0d8-6c3b-4773-b94e-a653d199d070 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a79c4edc8b9640f3b3e673e72728f601'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:16.086346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569daf98-b6c7-4fa5-bbcb-d9f93fee825f None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:16.091020 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212e64c-c2a6-4ca0-b376-7146fca8dbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:16.091599 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212e64c-c2a6-4ca0-b376-7146fca8dbe7 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:16.091876 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212e64c-c2a6-4ca0-b376-7146fca8dbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:16.092141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212e64c-c2a6-4ca0-b376-7146fca8dbe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:16.092763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212e64c-c2a6-4ca0-b376-7146fca8dbe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:16.094366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212e64c-c2a6-4ca0-b376-7146fca8dbe7 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:16.103906 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 875/1781] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:16.104641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212e64c-c2a6-4ca0-b376-7146fca8dbe7 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:16.114048 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212e64c-c2a6-4ca0-b376-7146fca8dbe7 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:16.118476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9dec2c-2567-40f1-b515-d9691554501f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:16.119013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dec2c-2567-40f1-b515-d9691554501f admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:16.119287 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dec2c-2567-40f1-b515-d9691554501f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:16.119550 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dec2c-2567-40f1-b515-d9691554501f admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:16.120139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9dec2c-2567-40f1-b515-d9691554501f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:16.121967 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9dec2c-2567-40f1-b515-d9691554501f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:16.165457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3c88d0d8-6c3b-4773-b94e-a653d199d070 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a79c4edc8b9640f3b3e673e72728f601'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:16.179315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3c88d0d8-6c3b-4773-b94e-a653d199d070 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a79c4edc8b9640f3b3e673e72728f601'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:16.180115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3c88d0d8-6c3b-4773-b94e-a653d199d070 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a79c4edc8b9640f3b3e673e72728f601'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:16.180753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3c88d0d8-6c3b-4773-b94e-a653d199d070 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a79c4edc8b9640f3b3e673e72728f601'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:16.193544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 907/1782] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:15 2022] DELETE /identity/v3/users/a79c4edc8b9640f3b3e673e72728f601 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:16.199190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88d0d8-6c3b-4773-b94e-a653d199d070 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:16.199190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3c88d0d8-6c3b-4773-b94e-a653d199d070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:16.251047 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 876/1783] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:16 2022] POST /identity/v3/projects => generated 389 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:16.255174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9dec2c-2567-40f1-b515-d9691554501f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:16.256991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6f9dec2c-2567-40f1-b515-d9691554501f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:16.276947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88d0d8-6c3b-4773-b94e-a653d199d070 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:16.334439 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9dec2c-2567-40f1-b515-d9691554501f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:16.341002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64aded-2383-450d-a4d1-e3fc2b414eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:16.341665 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64aded-2383-450d-a4d1-e3fc2b414eba admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:16.342105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64aded-2383-450d-a4d1-e3fc2b414eba admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:16.342496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64aded-2383-450d-a4d1-e3fc2b414eba admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:16.343233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64aded-2383-450d-a4d1-e3fc2b414eba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:16.346628 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd64aded-2383-450d-a4d1-e3fc2b414eba admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:16.346790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8817af-a703-4d2f-bcfe-949870afe591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:16.347479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8817af-a703-4d2f-bcfe-949870afe591 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:16.347769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8817af-a703-4d2f-bcfe-949870afe591 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:16.348386 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8817af-a703-4d2f-bcfe-949870afe591 admin admin] PATH_INFO: `/v3/projects/4ac36c09b5da4d0aa37782837e755bd6/users/11251f427585443ca190b9d242d21728/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:16.366377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8817af-a703-4d2f-bcfe-949870afe591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ac36c09b5da4d0aa37782837e755bd6, user_id=11251f427585443ca190b9d242d21728, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:16.369470 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8817af-a703-4d2f-bcfe-949870afe591 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:16.470350 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 908/1784] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:16 2022] PUT /identity/v3/projects/4ac36c09b5da4d0aa37782837e755bd6/users/11251f427585443ca190b9d242d21728/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:16.480560 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15ce15-fc66-4182-b583-e23ca8e8521e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:16.480839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15ce15-fc66-4182-b583-e23ca8e8521e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:16.481111 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a15ce15-fc66-4182-b583-e23ca8e8521e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:16.508612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7a15ce15-fc66-4182-b583-e23ca8e8521e None None] MFA Rules not processed for user `11251f427585443ca190b9d242d21728`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:16.509823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7a15ce15-fc66-4182-b583-e23ca8e8521e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:16.543240 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 877/1785] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:16 2022] POST /identity/v3/users => generated 301 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:56:16.544673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64aded-2383-450d-a4d1-e3fc2b414eba admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:16.546047 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-fd64aded-2383-450d-a4d1-e3fc2b414eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:16.574251 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64aded-2383-450d-a4d1-e3fc2b414eba admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:16.575826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 909/1786] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:16.580468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15ce15-fc66-4182-b583-e23ca8e8521e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:16.632647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:16.633285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:16.633563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:16.633854 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] PATH_INFO: `/v3/projects/64f709ba93664bcabd7267318a071765` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:16.635064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] RBAC: Authorizing `identity:delete_project(project_id=64f709ba93664bcabd7267318a071765)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:16.637146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:16.643007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a15ce15-fc66-4182-b583-e23ca8e8521e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:16.647545 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237cd1e-8a3d-4e8f-8c00-89d045b46db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:16.650364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237cd1e-8a3d-4e8f-8c00-89d045b46db5 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:16.650364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237cd1e-8a3d-4e8f-8c00-89d045b46db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:16.650364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a237cd1e-8a3d-4e8f-8c00-89d045b46db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:16.650364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237cd1e-8a3d-4e8f-8c00-89d045b46db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:16.650891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a237cd1e-8a3d-4e8f-8c00-89d045b46db5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:16.660908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 910/1787] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:16.661709 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237cd1e-8a3d-4e8f-8c00-89d045b46db5 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:16.674950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a237cd1e-8a3d-4e8f-8c00-89d045b46db5 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:16.679850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5726a3c-c5d9-41f6-9312-d23d20b5ce23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:16.680568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5726a3c-c5d9-41f6-9312-d23d20b5ce23 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:16.680846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5726a3c-c5d9-41f6-9312-d23d20b5ce23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:16.681113 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5726a3c-c5d9-41f6-9312-d23d20b5ce23 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:16.681774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5726a3c-c5d9-41f6-9312-d23d20b5ce23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:16.683937 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5726a3c-c5d9-41f6-9312-d23d20b5ce23 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:16.744458 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64f709ba93664bcabd7267318a071765', 'request_id': 'req-625b51d0-9024-4628-ae5c-ff0f66a4712c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:16.749919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64f709ba93664bcabd7267318a071765', 'request_id': 'req-625b51d0-9024-4628-ae5c-ff0f66a4712c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:16.756197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64f709ba93664bcabd7267318a071765', 'request_id': 'req-625b51d0-9024-4628-ae5c-ff0f66a4712c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:16.797316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 911/1788] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:16 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:16.798209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5726a3c-c5d9-41f6-9312-d23d20b5ce23 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:16.870902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] The token cache is being invalidate because project 64f709ba93664bcabd7267318a071765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:16.871199 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:16.871844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:16.873696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 878/1789] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:16 2022] DELETE /identity/v3/projects/64f709ba93664bcabd7267318a071765 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:16.874592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:16.876486 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:16.889108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5726a3c-c5d9-41f6-9312-d23d20b5ce23 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:16.894758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f627664-e94c-41ca-bf64-fb093003cb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:16.895518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f627664-e94c-41ca-bf64-fb093003cb47 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:16.895866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f627664-e94c-41ca-bf64-fb093003cb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:16.896213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f627664-e94c-41ca-bf64-fb093003cb47 admin admin] PATH_INFO: `/v3/projects/9c95d37c7371424a9a29b9c13c98ec52/users/6a02568accad4664bd93d3c7b3ff3174/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:16.911743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f627664-e94c-41ca-bf64-fb093003cb47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c95d37c7371424a9a29b9c13c98ec52, user_id=6a02568accad4664bd93d3c7b3ff3174, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:16.914939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f627664-e94c-41ca-bf64-fb093003cb47 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:16.946784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b51d0-9024-4628-ae5c-ff0f66a4712c admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:16.952097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86996e1-a9c9-476f-b952-3daaa86ea1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:16.954516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86996e1-a9c9-476f-b952-3daaa86ea1de admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:16.954921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86996e1-a9c9-476f-b952-3daaa86ea1de admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:16.955316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86996e1-a9c9-476f-b952-3daaa86ea1de admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:16.956136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86996e1-a9c9-476f-b952-3daaa86ea1de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:16.958207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86996e1-a9c9-476f-b952-3daaa86ea1de admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:17.007914 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 912/1790] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:16 2022] PUT /identity/v3/projects/9c95d37c7371424a9a29b9c13c98ec52/users/6a02568accad4664bd93d3c7b3ff3174/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:17.010463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f627664-e94c-41ca-bf64-fb093003cb47 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:17.014032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:17.018246 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5f627664-e94c-41ca-bf64-fb093003cb47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:17.048527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 879/1791] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:16 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:17.049072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86996e1-a9c9-476f-b952-3daaa86ea1de admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:17.052187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-f86996e1-a9c9-476f-b952-3daaa86ea1de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:17.068747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86996e1-a9c9-476f-b952-3daaa86ea1de admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:17.094120 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f627664-e94c-41ca-bf64-fb093003cb47 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:17.099319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621c5c5-a860-4c5f-8e4f-ed31fdfdb0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:17.099957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621c5c5-a860-4c5f-8e4f-ed31fdfdb0ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:17.100240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621c5c5-a860-4c5f-8e4f-ed31fdfdb0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:17.100511 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621c5c5-a860-4c5f-8e4f-ed31fdfdb0ca admin admin] PATH_INFO: `/v3/users/9b60c43c67a842d6bb40d285b40ed5d9` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:17.101646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621c5c5-a860-4c5f-8e4f-ed31fdfdb0ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b60c43c67a842d6bb40d285b40ed5d9)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:17.103634 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621c5c5-a860-4c5f-8e4f-ed31fdfdb0ca admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:17.138675 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:17.156990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e62f5-471a-4aee-85f7-63cc73488e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:17.156990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e62f5-471a-4aee-85f7-63cc73488e0e admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:17.156990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e62f5-471a-4aee-85f7-63cc73488e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:17.156990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e62f5-471a-4aee-85f7-63cc73488e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:17.157423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e62f5-471a-4aee-85f7-63cc73488e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:17.157423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e62f5-471a-4aee-85f7-63cc73488e0e admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:17.163968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 880/1792] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:17.164750 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e62f5-471a-4aee-85f7-63cc73488e0e admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:17.174446 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e62f5-471a-4aee-85f7-63cc73488e0e admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:17.179144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe406d5-32fd-4f66-8d70-3943b8b91e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:17.179692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe406d5-32fd-4f66-8d70-3943b8b91e5a admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:17.179972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe406d5-32fd-4f66-8d70-3943b8b91e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:17.185752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe406d5-32fd-4f66-8d70-3943b8b91e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:17.185752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe406d5-32fd-4f66-8d70-3943b8b91e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:17.185752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe406d5-32fd-4f66-8d70-3943b8b91e5a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:17.192763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 881/1793] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:17.193503 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe406d5-32fd-4f66-8d70-3943b8b91e5a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:17.203053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe406d5-32fd-4f66-8d70-3943b8b91e5a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:17.208436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f24f4-60b7-416c-87e1-41e774200f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:17.209179 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f24f4-60b7-416c-87e1-41e774200f03 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:17.209354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f24f4-60b7-416c-87e1-41e774200f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:17.209625 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f24f4-60b7-416c-87e1-41e774200f03 admin admin] PATH_INFO: `/v3/projects/9c95d37c7371424a9a29b9c13c98ec52/users/6a02568accad4664bd93d3c7b3ff3174/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:17.211639 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f24f4-60b7-416c-87e1-41e774200f03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c95d37c7371424a9a29b9c13c98ec52, user_id=6a02568accad4664bd93d3c7b3ff3174, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:17.214379 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f24f4-60b7-416c-87e1-41e774200f03 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:17.227049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 882/1794] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:17 2022] PUT /identity/v3/projects/9c95d37c7371424a9a29b9c13c98ec52/users/6a02568accad4664bd93d3c7b3ff3174/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:17.227698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f24f4-60b7-416c-87e1-41e774200f03 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:17.297082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f24f4-60b7-416c-87e1-41e774200f03 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:17.302994 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586abf68-8e93-48d8-8be2-7de01e6c3c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:17.303725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586abf68-8e93-48d8-8be2-7de01e6c3c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:17.304078 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586abf68-8e93-48d8-8be2-7de01e6c3c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:17.304355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586abf68-8e93-48d8-8be2-7de01e6c3c2a admin admin] PATH_INFO: `/v3/projects/d378169677094294a58756721b74487c/users/fede191a176f4fa1b0d660345e094408/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:17.324895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7621c5c5-a860-4c5f-8e4f-ed31fdfdb0ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b60c43c67a842d6bb40d285b40ed5d9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:17.325132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586abf68-8e93-48d8-8be2-7de01e6c3c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d378169677094294a58756721b74487c, user_id=fede191a176f4fa1b0d660345e094408, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:17.327918 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586abf68-8e93-48d8-8be2-7de01e6c3c2a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:17.435925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7621c5c5-a860-4c5f-8e4f-ed31fdfdb0ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b60c43c67a842d6bb40d285b40ed5d9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:17.445912 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 883/1795] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:17 2022] PUT /identity/v3/projects/d378169677094294a58756721b74487c/users/fede191a176f4fa1b0d660345e094408/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:17.447391 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a0437-d55c-49ce-b5a2-01131e1679ce None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:17.447673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a0437-d55c-49ce-b5a2-01131e1679ce None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:17.447962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a0437-d55c-49ce-b5a2-01131e1679ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:17.457742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7621c5c5-a860-4c5f-8e4f-ed31fdfdb0ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b60c43c67a842d6bb40d285b40ed5d9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:17.459550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7621c5c5-a860-4c5f-8e4f-ed31fdfdb0ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b60c43c67a842d6bb40d285b40ed5d9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:17.460337 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7621c5c5-a860-4c5f-8e4f-ed31fdfdb0ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b60c43c67a842d6bb40d285b40ed5d9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:17.481293 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 913/1796] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:17 2022] DELETE /identity/v3/users/9b60c43c67a842d6bb40d285b40ed5d9 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:17.485628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621c5c5-a860-4c5f-8e4f-ed31fdfdb0ca admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:17.487197 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7621c5c5-a860-4c5f-8e4f-ed31fdfdb0ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:17.498364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-800a0437-d55c-49ce-b5a2-01131e1679ce None None] MFA Rules not processed for user `6a02568accad4664bd93d3c7b3ff3174`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:17.499743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-800a0437-d55c-49ce-b5a2-01131e1679ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:17.594686 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621c5c5-a860-4c5f-8e4f-ed31fdfdb0ca admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:17.594776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 884/1797] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:17 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:17.597685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbee69d-3654-40f4-a915-e48581fa342f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:17.598671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbee69d-3654-40f4-a915-e48581fa342f admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:17.598671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbee69d-3654-40f4-a915-e48581fa342f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:17.598932 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbee69d-3654-40f4-a915-e48581fa342f admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:17.599574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbee69d-3654-40f4-a915-e48581fa342f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:17.601190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbee69d-3654-40f4-a915-e48581fa342f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:17.604255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a0437-d55c-49ce-b5a2-01131e1679ce None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:17.605661 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-800a0437-d55c-49ce-b5a2-01131e1679ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:17.615517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 914/1798] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:17.616976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbee69d-3654-40f4-a915-e48581fa342f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:17.627962 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbee69d-3654-40f4-a915-e48581fa342f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:17.633070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7800b0-e7c8-4190-92a9-910e5625f2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:17.634088 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7800b0-e7c8-4190-92a9-910e5625f2d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:17.634088 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7800b0-e7c8-4190-92a9-910e5625f2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:17.634276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7800b0-e7c8-4190-92a9-910e5625f2d6 admin admin] PATH_INFO: `/v3/projects/d378169677094294a58756721b74487c/users/fede191a176f4fa1b0d660345e094408/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:17.636528 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7800b0-e7c8-4190-92a9-910e5625f2d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d378169677094294a58756721b74487c, user_id=fede191a176f4fa1b0d660345e094408, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:17.639839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7800b0-e7c8-4190-92a9-910e5625f2d6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:17.651102 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 915/1799] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:17 2022] PUT /identity/v3/projects/d378169677094294a58756721b74487c/users/fede191a176f4fa1b0d660345e094408/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:17.664243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fdb00-d5f3-4881-9f26-66b873992aaf None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:17.664243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fdb00-d5f3-4881-9f26-66b873992aaf None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:17.664541 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fdb00-d5f3-4881-9f26-66b873992aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:17.668039 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a0437-d55c-49ce-b5a2-01131e1679ce None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:17.692792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f13fdb00-d5f3-4881-9f26-66b873992aaf None None] MFA Rules not processed for user `fede191a176f4fa1b0d660345e094408`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:17.694098 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f13fdb00-d5f3-4881-9f26-66b873992aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:17.720155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323d6a4-e83e-48eb-b571-d8a28ee81052 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:17.720743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323d6a4-e83e-48eb-b571-d8a28ee81052 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:17.721019 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323d6a4-e83e-48eb-b571-d8a28ee81052 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:17.721287 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9323d6a4-e83e-48eb-b571-d8a28ee81052 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:17.763756 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 916/1800] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:17.774785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fdb00-d5f3-4881-9f26-66b873992aaf None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:17.776468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f13fdb00-d5f3-4881-9f26-66b873992aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:17.784584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323d6a4-e83e-48eb-b571-d8a28ee81052 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:17.786099 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9323d6a4-e83e-48eb-b571-d8a28ee81052 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:17.788981 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fdb00-d5f3-4881-9f26-66b873992aaf None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:17.792605 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc8da4-57d6-44e0-b7a0-768e5b98c9cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:17.793223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc8da4-57d6-44e0-b7a0-768e5b98c9cd service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:17.793500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc8da4-57d6-44e0-b7a0-768e5b98c9cd service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:17.793791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cc8da4-57d6-44e0-b7a0-768e5b98c9cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:17.795434 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 885/1801] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:17 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:17.810609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc8da4-57d6-44e0-b7a0-768e5b98c9cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:17.812434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cc8da4-57d6-44e0-b7a0-768e5b98c9cd service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:17.823206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 917/1802] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:56:17.902190 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323d6a4-e83e-48eb-b571-d8a28ee81052 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:17.907965 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-9323d6a4-e83e-48eb-b571-d8a28ee81052 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:17.971819 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc8da4-57d6-44e0-b7a0-768e5b98c9cd service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:17.974209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-03cc8da4-57d6-44e0-b7a0-768e5b98c9cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:18.095180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9323d6a4-e83e-48eb-b571-d8a28ee81052 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:18.103056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:18.107561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cc8da4-57d6-44e0-b7a0-768e5b98c9cd service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:18.112946 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f04343-6cad-41b9-a19b-040b6d4348ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:18.113708 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f04343-6cad-41b9-a19b-040b6d4348ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:18.114109 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f04343-6cad-41b9-a19b-040b6d4348ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:18.114472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f04343-6cad-41b9-a19b-040b6d4348ec admin admin] PATH_INFO: `/v3/users/bc5f987003204c99929ab763ecbb6a0e` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:18.114981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8df59b0-f44e-483d-a1c5-9b1d9473232a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:18.115635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df59b0-f44e-483d-a1c5-9b1d9473232a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:18.115833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f04343-6cad-41b9-a19b-040b6d4348ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc5f987003204c99929ab763ecbb6a0e)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:18.115926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df59b0-f44e-483d-a1c5-9b1d9473232a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:18.116169 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8df59b0-f44e-483d-a1c5-9b1d9473232a admin admin] PATH_INFO: `/v3/projects/58498d0c2ed14f99bf5b98e1a9045775` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:18.118096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f04343-6cad-41b9-a19b-040b6d4348ec admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:18.118268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df59b0-f44e-483d-a1c5-9b1d9473232a admin admin] RBAC: Authorizing `identity:delete_project(project_id=58498d0c2ed14f99bf5b98e1a9045775)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:18.120248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8df59b0-f44e-483d-a1c5-9b1d9473232a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:18.402400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02f04343-6cad-41b9-a19b-040b6d4348ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc5f987003204c99929ab763ecbb6a0e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:18.402834 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a8df59b0-f44e-483d-a1c5-9b1d9473232a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58498d0c2ed14f99bf5b98e1a9045775', 'request_id': 'req-a8df59b0-f44e-483d-a1c5-9b1d9473232a'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:18.409836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a8df59b0-f44e-483d-a1c5-9b1d9473232a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58498d0c2ed14f99bf5b98e1a9045775', 'request_id': 'req-a8df59b0-f44e-483d-a1c5-9b1d9473232a'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:18.416188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a8df59b0-f44e-483d-a1c5-9b1d9473232a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58498d0c2ed14f99bf5b98e1a9045775', 'request_id': 'req-a8df59b0-f44e-483d-a1c5-9b1d9473232a'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:18.533006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02f04343-6cad-41b9-a19b-040b6d4348ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc5f987003204c99929ab763ecbb6a0e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:18.545623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02f04343-6cad-41b9-a19b-040b6d4348ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc5f987003204c99929ab763ecbb6a0e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:18.546299 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02f04343-6cad-41b9-a19b-040b6d4348ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc5f987003204c99929ab763ecbb6a0e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:18.546911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02f04343-6cad-41b9-a19b-040b6d4348ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc5f987003204c99929ab763ecbb6a0e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:18.559838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 918/1803] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:17 2022] DELETE /identity/v3/users/bc5f987003204c99929ab763ecbb6a0e => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:18.571207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f04343-6cad-41b9-a19b-040b6d4348ec admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:18.571207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-02f04343-6cad-41b9-a19b-040b6d4348ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:18.618720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f04343-6cad-41b9-a19b-040b6d4348ec admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:18.622656 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cde44-ceca-4a7c-b111-9c215c0b0a16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:18.623388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cde44-ceca-4a7c-b111-9c215c0b0a16 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:18.623515 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cde44-ceca-4a7c-b111-9c215c0b0a16 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:18.623918 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cde44-ceca-4a7c-b111-9c215c0b0a16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:18.625758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a8df59b0-f44e-483d-a1c5-9b1d9473232a admin admin] The token cache is being invalidate because project 58498d0c2ed14f99bf5b98e1a9045775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:18.625995 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ce9cde44-ceca-4a7c-b111-9c215c0b0a16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:18.626255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a8df59b0-f44e-483d-a1c5-9b1d9473232a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:18.626949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a8df59b0-f44e-483d-a1c5-9b1d9473232a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:18.628795 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 886/1804] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:17 2022] DELETE /identity/v3/projects/58498d0c2ed14f99bf5b98e1a9045775 => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:18.631201 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328104f7-74d4-4404-8036-2d684d5a844d None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:18.631201 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328104f7-74d4-4404-8036-2d684d5a844d None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:18.631201 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328104f7-74d4-4404-8036-2d684d5a844d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:18.652843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-328104f7-74d4-4404-8036-2d684d5a844d None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:18.654228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-328104f7-74d4-4404-8036-2d684d5a844d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:18.704491 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cde44-ceca-4a7c-b111-9c215c0b0a16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:18.706228 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cde44-ceca-4a7c-b111-9c215c0b0a16 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:18.716138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 919/1805] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:18.717395 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9d6af-8f09-495a-baec-1bf963932d31 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:18.717670 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9d6af-8f09-495a-baec-1bf963932d31 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:18.717991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9d6af-8f09-495a-baec-1bf963932d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:18.738902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 887/1806] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:18.740684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0bb9d6af-8f09-495a-baec-1bf963932d31 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:18.743433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:18.744103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0bb9d6af-8f09-495a-baec-1bf963932d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:18.747380 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328104f7-74d4-4404-8036-2d684d5a844d None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:18.751639 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-328104f7-74d4-4404-8036-2d684d5a844d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:18.755140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 920/1807] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:18.763258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13803b-6cf1-44c7-8c44-6707ad84bc79 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:18.763546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13803b-6cf1-44c7-8c44-6707ad84bc79 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:18.763873 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b13803b-6cf1-44c7-8c44-6707ad84bc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:18.791875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8b13803b-6cf1-44c7-8c44-6707ad84bc79 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:18.792425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8b13803b-6cf1-44c7-8c44-6707ad84bc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:18.816546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 921/1808] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:18.823395 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13803b-6cf1-44c7-8c44-6707ad84bc79 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:18.827499 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328104f7-74d4-4404-8036-2d684d5a844d None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:18.832748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a366a83-deae-439d-8f49-4a0d19b3ea9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:18.833489 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b13803b-6cf1-44c7-8c44-6707ad84bc79 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:18.833633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a366a83-deae-439d-8f49-4a0d19b3ea9c service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:18.834203 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a366a83-deae-439d-8f49-4a0d19b3ea9c service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:18.834520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a366a83-deae-439d-8f49-4a0d19b3ea9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:18.838198 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364d4c7-2026-4b3c-a747-df1979449833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:18.838810 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364d4c7-2026-4b3c-a747-df1979449833 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:18.839126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364d4c7-2026-4b3c-a747-df1979449833 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:18.839407 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364d4c7-2026-4b3c-a747-df1979449833 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:18.839997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364d4c7-2026-4b3c-a747-df1979449833 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:18.840315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364d4c7-2026-4b3c-a747-df1979449833 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:18.841929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364d4c7-2026-4b3c-a747-df1979449833 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:18.846412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a366a83-deae-439d-8f49-4a0d19b3ea9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:18.849060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a366a83-deae-439d-8f49-4a0d19b3ea9c service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:18.857554 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 922/1809] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:56:18.858608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364d4c7-2026-4b3c-a747-df1979449833 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:18.865462 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 888/1810] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:18.876828 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364d4c7-2026-4b3c-a747-df1979449833 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:18.881603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bed5f6-584a-48af-8bdc-d4dea24720e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:18.882236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bed5f6-584a-48af-8bdc-d4dea24720e9 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:18.882509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bed5f6-584a-48af-8bdc-d4dea24720e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:18.882776 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bed5f6-584a-48af-8bdc-d4dea24720e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:18.883394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bed5f6-584a-48af-8bdc-d4dea24720e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:18.885363 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bed5f6-584a-48af-8bdc-d4dea24720e9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:18.997921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 923/1811] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:18 2022] POST /identity/v3/projects => generated 363 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:19.002694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a366a83-deae-439d-8f49-4a0d19b3ea9c service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:19.012709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a366a83-deae-439d-8f49-4a0d19b3ea9c service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:19.074209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332f8c4-999d-4574-81b5-94d799999a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:19.074209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332f8c4-999d-4574-81b5-94d799999a08 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:19.074209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332f8c4-999d-4574-81b5-94d799999a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:19.074209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0332f8c4-999d-4574-81b5-94d799999a08 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:19.074551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0332f8c4-999d-4574-81b5-94d799999a08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:19.076289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0332f8c4-999d-4574-81b5-94d799999a08 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:19.195726 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bed5f6-584a-48af-8bdc-d4dea24720e9 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:19.196566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 889/1812] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:19 2022] POST /identity/v3/users => generated 282 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:56:19.206909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bed5f6-584a-48af-8bdc-d4dea24720e9 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:19.211580 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b810149-f775-4e2e-bcf8-407619ef78ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:19.212723 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b810149-f775-4e2e-bcf8-407619ef78ba admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:19.212723 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b810149-f775-4e2e-bcf8-407619ef78ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:19.212994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b810149-f775-4e2e-bcf8-407619ef78ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:19.213750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b810149-f775-4e2e-bcf8-407619ef78ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:19.216085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b810149-f775-4e2e-bcf8-407619ef78ba admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:19.233160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332f8c4-999d-4574-81b5-94d799999a08 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:19.240639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 924/1813] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:19.245748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0332f8c4-999d-4574-81b5-94d799999a08 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:19.250620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b84c5d-a2e2-46eb-be9e-abe8663d8402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:19.251297 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b84c5d-a2e2-46eb-be9e-abe8663d8402 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:19.252258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b84c5d-a2e2-46eb-be9e-abe8663d8402 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:19.252258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b84c5d-a2e2-46eb-be9e-abe8663d8402 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:19.252565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b84c5d-a2e2-46eb-be9e-abe8663d8402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:19.254193 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b84c5d-a2e2-46eb-be9e-abe8663d8402 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:19.270103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b810149-f775-4e2e-bcf8-407619ef78ba admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:19.271002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 890/1814] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:19.280226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b810149-f775-4e2e-bcf8-407619ef78ba admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:19.293508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e23f12-32c9-4e1d-8fdb-5e027e1c2c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:19.294656 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23f12-32c9-4e1d-8fdb-5e027e1c2c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:19.295129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23f12-32c9-4e1d-8fdb-5e027e1c2c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:19.295586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e23f12-32c9-4e1d-8fdb-5e027e1c2c0b admin admin] PATH_INFO: `/v3/projects/1c91dcee759c4ca382274395975ede91/users/9bef888a851a4394aee4ea2987fc1b04/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:19.318155 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e23f12-32c9-4e1d-8fdb-5e027e1c2c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c91dcee759c4ca382274395975ede91, user_id=9bef888a851a4394aee4ea2987fc1b04, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:19.318640 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b84c5d-a2e2-46eb-be9e-abe8663d8402 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:19.320515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-42b84c5d-a2e2-46eb-be9e-abe8663d8402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:19.321511 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e23f12-32c9-4e1d-8fdb-5e027e1c2c0b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:19.334079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b84c5d-a2e2-46eb-be9e-abe8663d8402 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:19.339162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:19.340016 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:19.340358 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:19.340754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] PATH_INFO: `/v3/projects/0ef6c0779da24fd6af98228402011cfb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:19.342094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ef6c0779da24fd6af98228402011cfb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:19.344056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:19.397873 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 925/1815] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:19 2022] PUT /identity/v3/projects/1c91dcee759c4ca382274395975ede91/users/9bef888a851a4394aee4ea2987fc1b04/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:19.409252 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afa731-fbd2-4611-84f6-104b126542b6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:19.409846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afa731-fbd2-4611-84f6-104b126542b6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:19.410257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afa731-fbd2-4611-84f6-104b126542b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:19.449988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-22afa731-fbd2-4611-84f6-104b126542b6 None None] MFA Rules not processed for user `9bef888a851a4394aee4ea2987fc1b04`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:19.456523 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-22afa731-fbd2-4611-84f6-104b126542b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:19.487583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ef6c0779da24fd6af98228402011cfb', 'request_id': 'req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:19.498689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ef6c0779da24fd6af98228402011cfb', 'request_id': 'req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:19.507698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ef6c0779da24fd6af98228402011cfb', 'request_id': 'req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:19.566980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 926/1816] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:19.571704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afa731-fbd2-4611-84f6-104b126542b6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:19.618076 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] The token cache is being invalidate because project 0ef6c0779da24fd6af98228402011cfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:19.618475 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:19.619135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:19.620856 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 891/1817] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:19 2022] DELETE /identity/v3/projects/0ef6c0779da24fd6af98228402011cfb => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:19.624807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:19.626163 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:19.634108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afa731-fbd2-4611-84f6-104b126542b6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:19.638965 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f01eb-192f-4dc6-a780-79344fd8e89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:19.639095 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893b929-9ed7-4c8a-9fe3-7f8149956b7d admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:19.639567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f01eb-192f-4dc6-a780-79344fd8e89e admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:19.639844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f01eb-192f-4dc6-a780-79344fd8e89e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:19.640116 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f01eb-192f-4dc6-a780-79344fd8e89e admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:19.640740 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f01eb-192f-4dc6-a780-79344fd8e89e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:19.643195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f01eb-192f-4dc6-a780-79344fd8e89e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:19.645104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0757e4-e667-4a15-8d7a-251386557013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:19.646010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0757e4-e667-4a15-8d7a-251386557013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:19.646322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0757e4-e667-4a15-8d7a-251386557013 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:19.646591 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0757e4-e667-4a15-8d7a-251386557013 admin admin] PATH_INFO: `/v3/users/6a02568accad4664bd93d3c7b3ff3174` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:19.647752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0757e4-e667-4a15-8d7a-251386557013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a02568accad4664bd93d3c7b3ff3174)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:19.650015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0757e4-e667-4a15-8d7a-251386557013 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:19.780058 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 927/1818] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:19 2022] POST /identity/v3/projects => generated 363 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:56:19.783848 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f01eb-192f-4dc6-a780-79344fd8e89e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:19.785624 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f60f01eb-192f-4dc6-a780-79344fd8e89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:19.849452 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f01eb-192f-4dc6-a780-79344fd8e89e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:19.856183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8f4b8-ab7e-4116-837f-3b3f17da79c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:19.857006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8f4b8-ab7e-4116-837f-3b3f17da79c4 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:19.857287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8f4b8-ab7e-4116-837f-3b3f17da79c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:19.857566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8f4b8-ab7e-4116-837f-3b3f17da79c4 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:19.858226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8f4b8-ab7e-4116-837f-3b3f17da79c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:19.860244 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8f4b8-ab7e-4116-837f-3b3f17da79c4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:19.913736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6e0757e4-e667-4a15-8d7a-251386557013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a02568accad4664bd93d3c7b3ff3174'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:19.925854 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6e0757e4-e667-4a15-8d7a-251386557013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a02568accad4664bd93d3c7b3ff3174'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:20.006068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 928/1819] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:19 2022] POST /identity/v3/users => generated 288 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:56:20.011472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8f4b8-ab7e-4116-837f-3b3f17da79c4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:20.021343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8f4b8-ab7e-4116-837f-3b3f17da79c4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:20.025910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2f15d-d124-404e-8177-d76e071d2d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:20.026476 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2f15d-d124-404e-8177-d76e071d2d3d admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:20.026753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2f15d-d124-404e-8177-d76e071d2d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:20.027019 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2f15d-d124-404e-8177-d76e071d2d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:20.027628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2f15d-d124-404e-8177-d76e071d2d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:20.029184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2f15d-d124-404e-8177-d76e071d2d3d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:20.038749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 929/1820] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:56:20.041429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6e0757e4-e667-4a15-8d7a-251386557013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a02568accad4664bd93d3c7b3ff3174'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:20.042332 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6e0757e4-e667-4a15-8d7a-251386557013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a02568accad4664bd93d3c7b3ff3174'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:20.042599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2f15d-d124-404e-8177-d76e071d2d3d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:20.044153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a8e2f15d-d124-404e-8177-d76e071d2d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:20.049912 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6e0757e4-e667-4a15-8d7a-251386557013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a02568accad4664bd93d3c7b3ff3174'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:20.055995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 892/1821] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:19 2022] DELETE /identity/v3/users/6a02568accad4664bd93d3c7b3ff3174 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:20.064008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2f15d-d124-404e-8177-d76e071d2d3d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:20.115744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df33e6-f4fc-4816-b833-173a366ffd12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:20.116747 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df33e6-f4fc-4816-b833-173a366ffd12 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:20.116922 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df33e6-f4fc-4816-b833-173a366ffd12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:20.117228 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df33e6-f4fc-4816-b833-173a366ffd12 admin admin] PATH_INFO: `/v3/projects/28ae8020a87e4b83a9f9b6a4e04e0eca/users/6a3524d90de743dbaed6df366039d05f/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:20.132469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df33e6-f4fc-4816-b833-173a366ffd12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28ae8020a87e4b83a9f9b6a4e04e0eca, user_id=6a3524d90de743dbaed6df366039d05f, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:20.135181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5df33e6-f4fc-4816-b833-173a366ffd12 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:20.247758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 930/1822] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:20 2022] PUT /identity/v3/projects/28ae8020a87e4b83a9f9b6a4e04e0eca/users/6a3524d90de743dbaed6df366039d05f/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:20.253143 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0757e4-e667-4a15-8d7a-251386557013 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:20.259744 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:20.265093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0757e4-e667-4a15-8d7a-251386557013 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:20.329153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75cd3ed2-ba8e-470b-9d7c-2e17fb1bc9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:20.329823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd3ed2-ba8e-470b-9d7c-2e17fb1bc9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:20.330125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd3ed2-ba8e-470b-9d7c-2e17fb1bc9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:20.331610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cd3ed2-ba8e-470b-9d7c-2e17fb1bc9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:20.332263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd3ed2-ba8e-470b-9d7c-2e17fb1bc9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:20.333985 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75cd3ed2-ba8e-470b-9d7c-2e17fb1bc9f3 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:20.347517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 893/1823] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:20 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:20.348662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df33e6-f4fc-4816-b833-173a366ffd12 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:20.359630 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df33e6-f4fc-4816-b833-173a366ffd12 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:20.365108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f7818-cc31-46eb-a120-bf939e75037f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:20.365941 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f7818-cc31-46eb-a120-bf939e75037f admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:20.366420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f7818-cc31-46eb-a120-bf939e75037f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:20.366629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341f7818-cc31-46eb-a120-bf939e75037f admin admin] PATH_INFO: `/v3/projects/28ae8020a87e4b83a9f9b6a4e04e0eca/users/6a3524d90de743dbaed6df366039d05f/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:20.368584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f7818-cc31-46eb-a120-bf939e75037f admin admin] RBAC: Authorizing `identity:create_grant(project_id=28ae8020a87e4b83a9f9b6a4e04e0eca, user_id=6a3524d90de743dbaed6df366039d05f, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:20.371533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341f7818-cc31-46eb-a120-bf939e75037f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:20.382950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 931/1824] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:20 2022] PUT /identity/v3/projects/28ae8020a87e4b83a9f9b6a4e04e0eca/users/6a3524d90de743dbaed6df366039d05f/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:20.401927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8572c-c27c-4101-bfad-b1de4311688b None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:20.402221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8572c-c27c-4101-bfad-b1de4311688b None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:20.402495 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8572c-c27c-4101-bfad-b1de4311688b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:20.433305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-c9b8572c-c27c-4101-bfad-b1de4311688b None None] MFA Rules not processed for user `6a3524d90de743dbaed6df366039d05f`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:20.434577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c9b8572c-c27c-4101-bfad-b1de4311688b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:20.459720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:20.527452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 894/1825] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:20.536227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f7818-cc31-46eb-a120-bf939e75037f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:20.538154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-341f7818-cc31-46eb-a120-bf939e75037f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:20.587684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8572c-c27c-4101-bfad-b1de4311688b None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:20.590151 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c9b8572c-c27c-4101-bfad-b1de4311688b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:20.618864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341f7818-cc31-46eb-a120-bf939e75037f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:20.632738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0f1e3-f293-4d08-9fa1-84ea8b205597 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:20.633898 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0f1e3-f293-4d08-9fa1-84ea8b205597 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:20.634267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0f1e3-f293-4d08-9fa1-84ea8b205597 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:20.634637 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0f1e3-f293-4d08-9fa1-84ea8b205597 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:20.646163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0f1e3-f293-4d08-9fa1-84ea8b205597 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:20.647657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0f1e3-f293-4d08-9fa1-84ea8b205597 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:20.659096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 932/1826] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:20.682064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8572c-c27c-4101-bfad-b1de4311688b None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:20.689253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a773bc-3631-48bd-8492-9b9ea0d104ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:20.689950 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a773bc-3631-48bd-8492-9b9ea0d104ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:20.690663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a773bc-3631-48bd-8492-9b9ea0d104ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:20.690945 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a773bc-3631-48bd-8492-9b9ea0d104ce admin admin] PATH_INFO: `/v3/projects/9c95d37c7371424a9a29b9c13c98ec52` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:20.692289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a773bc-3631-48bd-8492-9b9ea0d104ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c95d37c7371424a9a29b9c13c98ec52)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:20.694802 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a773bc-3631-48bd-8492-9b9ea0d104ce admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:20.819264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-23a773bc-3631-48bd-8492-9b9ea0d104ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c95d37c7371424a9a29b9c13c98ec52', 'request_id': 'req-23a773bc-3631-48bd-8492-9b9ea0d104ce'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:20.824652 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-23a773bc-3631-48bd-8492-9b9ea0d104ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c95d37c7371424a9a29b9c13c98ec52', 'request_id': 'req-23a773bc-3631-48bd-8492-9b9ea0d104ce'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:20.831320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-23a773bc-3631-48bd-8492-9b9ea0d104ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c95d37c7371424a9a29b9c13c98ec52', 'request_id': 'req-23a773bc-3631-48bd-8492-9b9ea0d104ce'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:20.850505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0f1e3-f293-4d08-9fa1-84ea8b205597 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:20.858787 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0f1e3-f293-4d08-9fa1-84ea8b205597 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:20.910715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93be8e4a-6fdd-4881-b034-7e117ad17dca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:20.911406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be8e4a-6fdd-4881-b034-7e117ad17dca service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:20.911682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be8e4a-6fdd-4881-b034-7e117ad17dca service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:20.911954 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93be8e4a-6fdd-4881-b034-7e117ad17dca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:20.914043 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-93be8e4a-6fdd-4881-b034-7e117ad17dca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:20.930750 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-23a773bc-3631-48bd-8492-9b9ea0d104ce admin admin] The token cache is being invalidate because project 9c95d37c7371424a9a29b9c13c98ec52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:20.933106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-23a773bc-3631-48bd-8492-9b9ea0d104ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:20.933106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-23a773bc-3631-48bd-8492-9b9ea0d104ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:20.936934 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 895/1827] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:20 2022] DELETE /identity/v3/projects/9c95d37c7371424a9a29b9c13c98ec52 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:20.940873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e51cd-619e-491b-a6c8-93521c705512 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:20.941155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e51cd-619e-491b-a6c8-93521c705512 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:20.941428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e51cd-619e-491b-a6c8-93521c705512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:20.963341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-9d1e51cd-619e-491b-a6c8-93521c705512 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:20.964587 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-9d1e51cd-619e-491b-a6c8-93521c705512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:20.981019 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be8e4a-6fdd-4881-b034-7e117ad17dca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:20.982493 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93be8e4a-6fdd-4881-b034-7e117ad17dca service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:20.993899 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 933/1828] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:56:21.046981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 896/1829] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:21.055008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d28386-ea83-4e10-9263-23281090d0d9 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:21.055288 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d28386-ea83-4e10-9263-23281090d0d9 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:21.059388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d28386-ea83-4e10-9263-23281090d0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:21.082674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-02d28386-ea83-4e10-9263-23281090d0d9 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:21.083955 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-02d28386-ea83-4e10-9263-23281090d0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:21.096131 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 934/1830] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:21.097431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e51cd-619e-491b-a6c8-93521c705512 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:21.107720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e51cd-619e-491b-a6c8-93521c705512 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:21.116528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933731fe-4c9e-4a42-a525-52d5b9907509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:21.117205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933731fe-4c9e-4a42-a525-52d5b9907509 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:21.117481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933731fe-4c9e-4a42-a525-52d5b9907509 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:21.117748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933731fe-4c9e-4a42-a525-52d5b9907509 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:21.118433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933731fe-4c9e-4a42-a525-52d5b9907509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:21.118775 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933731fe-4c9e-4a42-a525-52d5b9907509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:21.120954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933731fe-4c9e-4a42-a525-52d5b9907509 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:21.133148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 897/1831] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:56:21.137660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d28386-ea83-4e10-9263-23281090d0d9 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:21.152404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d28386-ea83-4e10-9263-23281090d0d9 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:21.163253 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788392a1-f199-424b-9278-544e7cd3c5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:21.163847 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788392a1-f199-424b-9278-544e7cd3c5eb admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:21.168970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788392a1-f199-424b-9278-544e7cd3c5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:21.168970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788392a1-f199-424b-9278-544e7cd3c5eb admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:21.168970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788392a1-f199-424b-9278-544e7cd3c5eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:21.168970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788392a1-f199-424b-9278-544e7cd3c5eb admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:21.253534 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 935/1832] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:21 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:21.255268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933731fe-4c9e-4a42-a525-52d5b9907509 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:21.266136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933731fe-4c9e-4a42-a525-52d5b9907509 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:21.324239 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7d367-023c-438d-977d-f2e19d1ab356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:21.324849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d367-023c-438d-977d-f2e19d1ab356 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:21.325128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d367-023c-438d-977d-f2e19d1ab356 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:21.325397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7d367-023c-438d-977d-f2e19d1ab356 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:21.326052 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7d367-023c-438d-977d-f2e19d1ab356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:21.327950 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7d367-023c-438d-977d-f2e19d1ab356 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:21.373537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:21.465564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 898/1833] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:21 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:21.469480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788392a1-f199-424b-9278-544e7cd3c5eb admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:21.481154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788392a1-f199-424b-9278-544e7cd3c5eb admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:21.486573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf278a-0e8b-42d4-8574-f6b4d59432e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:21.487194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf278a-0e8b-42d4-8574-f6b4d59432e2 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:21.487480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf278a-0e8b-42d4-8574-f6b4d59432e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:21.487756 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf278a-0e8b-42d4-8574-f6b4d59432e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:21.488514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf278a-0e8b-42d4-8574-f6b4d59432e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:21.490202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf278a-0e8b-42d4-8574-f6b4d59432e2 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:21.500662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 936/1834] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:56:21.504720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7d367-023c-438d-977d-f2e19d1ab356 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:21.515607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7d367-023c-438d-977d-f2e19d1ab356 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:21.520290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e064a4af-cffb-4aea-98ae-6abf7c18ddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:21.520941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064a4af-cffb-4aea-98ae-6abf7c18ddf5 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:21.521255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064a4af-cffb-4aea-98ae-6abf7c18ddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:21.521520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e064a4af-cffb-4aea-98ae-6abf7c18ddf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:21.522188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064a4af-cffb-4aea-98ae-6abf7c18ddf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:21.523778 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e064a4af-cffb-4aea-98ae-6abf7c18ddf5 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:21.535792 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 899/1835] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:21.544272 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf278a-0e8b-42d4-8574-f6b4d59432e2 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:21.554715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf278a-0e8b-42d4-8574-f6b4d59432e2 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:21.560197 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e4d06-0f2b-4249-a089-e943f8b5d192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:21.564316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e4d06-0f2b-4249-a089-e943f8b5d192 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:21.564316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e4d06-0f2b-4249-a089-e943f8b5d192 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:21.564316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e4d06-0f2b-4249-a089-e943f8b5d192 admin admin] PATH_INFO: `/v3/projects/0c46eccf65b5477c9a388c9b5644d819/users/5ad6412a0adb4dfcb38eeed52caee583/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:21.580608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e4d06-0f2b-4249-a089-e943f8b5d192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c46eccf65b5477c9a388c9b5644d819, user_id=5ad6412a0adb4dfcb38eeed52caee583, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:21.587326 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694e4d06-0f2b-4249-a089-e943f8b5d192 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:21.694279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 937/1836] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:21 2022] PUT /identity/v3/projects/0c46eccf65b5477c9a388c9b5644d819/users/5ad6412a0adb4dfcb38eeed52caee583/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:21.695752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c0864-2d57-45fc-8b8c-d27bf1475412 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:21.696069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c0864-2d57-45fc-8b8c-d27bf1475412 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:21.696369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c0864-2d57-45fc-8b8c-d27bf1475412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:21.731004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-5e5c0864-2d57-45fc-8b8c-d27bf1475412 None None] MFA Rules not processed for user `5ad6412a0adb4dfcb38eeed52caee583`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:21.732119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-5e5c0864-2d57-45fc-8b8c-d27bf1475412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:21.778839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:21.809848 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 900/1837] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:21.814399 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e4d06-0f2b-4249-a089-e943f8b5d192 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:21.902240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e4d06-0f2b-4249-a089-e943f8b5d192 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:21.902240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cafcc-ce8d-41e2-9742-0103d3c44346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:21.903085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cafcc-ce8d-41e2-9742-0103d3c44346 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:21.903183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cafcc-ce8d-41e2-9742-0103d3c44346 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:21.903591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cafcc-ce8d-41e2-9742-0103d3c44346 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:21.908760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cafcc-ce8d-41e2-9742-0103d3c44346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:21.908760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cafcc-ce8d-41e2-9742-0103d3c44346 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:22.064924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 938/1838] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:21 2022] POST /identity/v3/projects => generated 381 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:56:22.066094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c0864-2d57-45fc-8b8c-d27bf1475412 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:22.077347 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c0864-2d57-45fc-8b8c-d27bf1475412 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:22.140053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b781f8-02f7-4171-967a-dc5a61e69f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:22.140688 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b781f8-02f7-4171-967a-dc5a61e69f3c admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:22.140963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b781f8-02f7-4171-967a-dc5a61e69f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:22.141289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b781f8-02f7-4171-967a-dc5a61e69f3c admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:22.142072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b781f8-02f7-4171-967a-dc5a61e69f3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:22.144149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b781f8-02f7-4171-967a-dc5a61e69f3c admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:22.242567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 901/1839] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:22 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:22.246691 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cafcc-ce8d-41e2-9742-0103d3c44346 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:22.258878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cafcc-ce8d-41e2-9742-0103d3c44346 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:22.266825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c151b9-8ff1-4f6d-8400-6e35fd654a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:22.266825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c151b9-8ff1-4f6d-8400-6e35fd654a0d admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:22.266825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c151b9-8ff1-4f6d-8400-6e35fd654a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:22.266825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c151b9-8ff1-4f6d-8400-6e35fd654a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:22.272666 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c151b9-8ff1-4f6d-8400-6e35fd654a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:22.272666 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c151b9-8ff1-4f6d-8400-6e35fd654a0d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:22.283451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 939/1840] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:22.284689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b781f8-02f7-4171-967a-dc5a61e69f3c admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:22.295230 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b781f8-02f7-4171-967a-dc5a61e69f3c admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:22.310829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679844b1-3a34-4966-a8bc-281974a69bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:22.310829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679844b1-3a34-4966-a8bc-281974a69bdd admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:22.310829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679844b1-3a34-4966-a8bc-281974a69bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:22.310829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679844b1-3a34-4966-a8bc-281974a69bdd admin admin] PATH_INFO: `/v3/projects/49d46c13d33347aaa1886999a5e88454/users/fbb3cb80af9a4bbb99a810c1b2be3f8f/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:22.328794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679844b1-3a34-4966-a8bc-281974a69bdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d46c13d33347aaa1886999a5e88454, user_id=fbb3cb80af9a4bbb99a810c1b2be3f8f, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:22.331732 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679844b1-3a34-4966-a8bc-281974a69bdd admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:22.491393 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 902/1841] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:22 2022] PUT /identity/v3/projects/49d46c13d33347aaa1886999a5e88454/users/fbb3cb80af9a4bbb99a810c1b2be3f8f/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:22.492189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c151b9-8ff1-4f6d-8400-6e35fd654a0d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:22.503177 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c151b9-8ff1-4f6d-8400-6e35fd654a0d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:22.558472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883616c8-6f1c-454a-bb12-af4d28475789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:22.559077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883616c8-6f1c-454a-bb12-af4d28475789 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:22.559350 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883616c8-6f1c-454a-bb12-af4d28475789 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:22.559693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883616c8-6f1c-454a-bb12-af4d28475789 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:22.560410 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883616c8-6f1c-454a-bb12-af4d28475789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:22.562107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883616c8-6f1c-454a-bb12-af4d28475789 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:22.572089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 940/1842] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:22.576368 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679844b1-3a34-4966-a8bc-281974a69bdd admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:22.587465 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679844b1-3a34-4966-a8bc-281974a69bdd admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:22.592503 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cf730-e889-4323-9901-8bd23bc992a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:22.593096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cf730-e889-4323-9901-8bd23bc992a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:22.593371 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cf730-e889-4323-9901-8bd23bc992a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:22.593642 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6cf730-e889-4323-9901-8bd23bc992a4 admin admin] PATH_INFO: `/v3/projects/49d46c13d33347aaa1886999a5e88454/users/fbb3cb80af9a4bbb99a810c1b2be3f8f/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:22.605531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6cf730-e889-4323-9901-8bd23bc992a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d46c13d33347aaa1886999a5e88454, user_id=fbb3cb80af9a4bbb99a810c1b2be3f8f, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:22.605531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6cf730-e889-4323-9901-8bd23bc992a4 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:22.610080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 903/1843] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:22 2022] PUT /identity/v3/projects/49d46c13d33347aaa1886999a5e88454/users/fbb3cb80af9a4bbb99a810c1b2be3f8f/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:22.622539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2d429-5c6e-4bae-afb3-21447c6bb95b None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:22.622969 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2d429-5c6e-4bae-afb3-21447c6bb95b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:22.623397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b2d429-5c6e-4bae-afb3-21447c6bb95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:22.654281 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-16b2d429-5c6e-4bae-afb3-21447c6bb95b None None] MFA Rules not processed for user `fbb3cb80af9a4bbb99a810c1b2be3f8f`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:22.655679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-16b2d429-5c6e-4bae-afb3-21447c6bb95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:22.743546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 941/1844] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:22.821880 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cf730-e889-4323-9901-8bd23bc992a4 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:22.823698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-db6cf730-e889-4323-9901-8bd23bc992a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:22.902870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6cf730-e889-4323-9901-8bd23bc992a4 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:22.905396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c52af-e55c-45fe-b560-b0c1b3313582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:22.906070 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c52af-e55c-45fe-b560-b0c1b3313582 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:22.907813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c52af-e55c-45fe-b560-b0c1b3313582 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:22.908282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c52af-e55c-45fe-b560-b0c1b3313582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:22.944428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c52af-e55c-45fe-b560-b0c1b3313582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:22.946057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c52af-e55c-45fe-b560-b0c1b3313582 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:22.970391 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 904/1845] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:22 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:23.185905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2d429-5c6e-4bae-afb3-21447c6bb95b None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:23.187397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-16b2d429-5c6e-4bae-afb3-21447c6bb95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:23.293871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:23.331551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b2d429-5c6e-4bae-afb3-21447c6bb95b None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:23.336624 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee9c73-e1a9-4268-87e3-0a756177529b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:23.337351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee9c73-e1a9-4268-87e3-0a756177529b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:23.337671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee9c73-e1a9-4268-87e3-0a756177529b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:23.337976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee9c73-e1a9-4268-87e3-0a756177529b admin admin] PATH_INFO: `/v3/users/11251f427585443ca190b9d242d21728` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:23.342444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee9c73-e1a9-4268-87e3-0a756177529b admin admin] RBAC: Authorizing `identity:delete_user(user_id=11251f427585443ca190b9d242d21728)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:23.344769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee9c73-e1a9-4268-87e3-0a756177529b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:23.415904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f261d75-3d37-4065-9698-b38d8d9016b5 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:23.416320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f261d75-3d37-4065-9698-b38d8d9016b5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:23.416320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f261d75-3d37-4065-9698-b38d8d9016b5 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:23.417646 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 905/1846] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 19:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:56:23.425286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df463dd8-ec2a-428a-b7d6-e4a4289a5112 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:23.425394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df463dd8-ec2a-428a-b7d6-e4a4289a5112 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:23.425664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df463dd8-ec2a-428a-b7d6-e4a4289a5112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:23.469421 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-df463dd8-ec2a-428a-b7d6-e4a4289a5112 None None] MFA Rules not processed for user `2a4284b59f804dc59674d621d87634bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:23.471715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-df463dd8-ec2a-428a-b7d6-e4a4289a5112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:23.624736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 906/1847] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 19:56:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:23.628891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5aee9c73-e1a9-4268-87e3-0a756177529b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11251f427585443ca190b9d242d21728'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:23.640026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df463dd8-ec2a-428a-b7d6-e4a4289a5112 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:23.640501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-df463dd8-ec2a-428a-b7d6-e4a4289a5112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:23.659613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df463dd8-ec2a-428a-b7d6-e4a4289a5112 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:23.668581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6005ec-0a6d-4f2f-92d9-75a69e67f528 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:23.668581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6005ec-0a6d-4f2f-92d9-75a69e67f528 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:23.668581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6005ec-0a6d-4f2f-92d9-75a69e67f528 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:23.668581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6005ec-0a6d-4f2f-92d9-75a69e67f528 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:23.678012 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6005ec-0a6d-4f2f-92d9-75a69e67f528 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:23.680045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6005ec-0a6d-4f2f-92d9-75a69e67f528 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:23.693295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 907/1848] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:56:23.740872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5aee9c73-e1a9-4268-87e3-0a756177529b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11251f427585443ca190b9d242d21728'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:23.773297 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5aee9c73-e1a9-4268-87e3-0a756177529b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11251f427585443ca190b9d242d21728'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:23.774242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5aee9c73-e1a9-4268-87e3-0a756177529b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11251f427585443ca190b9d242d21728'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:23.775014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5aee9c73-e1a9-4268-87e3-0a756177529b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11251f427585443ca190b9d242d21728'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:23.840480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 942/1849] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:23 2022] DELETE /identity/v3/users/11251f427585443ca190b9d242d21728 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:23.842207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b34c96-8aec-4580-8539-498c6a2b48e1 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:23.842713 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b34c96-8aec-4580-8539-498c6a2b48e1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:23.843362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b34c96-8aec-4580-8539-498c6a2b48e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:23.898070 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-80b34c96-8aec-4580-8539-498c6a2b48e1 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:23.898070 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-80b34c96-8aec-4580-8539-498c6a2b48e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:24.007726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 908/1850] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:24.021437 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee9c73-e1a9-4268-87e3-0a756177529b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:24.022824 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5aee9c73-e1a9-4268-87e3-0a756177529b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:24.088092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee9c73-e1a9-4268-87e3-0a756177529b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:24.092438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b436b-c6c4-4cbf-a9bc-73607c0acc37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:24.093121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b436b-c6c4-4cbf-a9bc-73607c0acc37 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:24.093395 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b436b-c6c4-4cbf-a9bc-73607c0acc37 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:24.093721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b436b-c6c4-4cbf-a9bc-73607c0acc37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:24.104782 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b436b-c6c4-4cbf-a9bc-73607c0acc37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:24.106319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b436b-c6c4-4cbf-a9bc-73607c0acc37 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:24.123699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 943/1851] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:24.204315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b34c96-8aec-4580-8539-498c6a2b48e1 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:24.206785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-80b34c96-8aec-4580-8539-498c6a2b48e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:24.257657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b34c96-8aec-4580-8539-498c6a2b48e1 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:24.273734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f1023-4e77-476b-9ecd-d81ca1769edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:24.275017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f1023-4e77-476b-9ecd-d81ca1769edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:24.275505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f1023-4e77-476b-9ecd-d81ca1769edf admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:24.276125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f1023-4e77-476b-9ecd-d81ca1769edf admin admin] PATH_INFO: `/v3/users/5ad6412a0adb4dfcb38eeed52caee583` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:24.294355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f1023-4e77-476b-9ecd-d81ca1769edf admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ad6412a0adb4dfcb38eeed52caee583)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:24.299616 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f1023-4e77-476b-9ecd-d81ca1769edf admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:24.592406 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0e4f1023-4e77-476b-9ecd-d81ca1769edf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ad6412a0adb4dfcb38eeed52caee583'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:24.612462 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0e4f1023-4e77-476b-9ecd-d81ca1769edf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ad6412a0adb4dfcb38eeed52caee583'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:24.693256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0e4f1023-4e77-476b-9ecd-d81ca1769edf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ad6412a0adb4dfcb38eeed52caee583'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:24.693999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0e4f1023-4e77-476b-9ecd-d81ca1769edf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ad6412a0adb4dfcb38eeed52caee583'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:24.702855 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0e4f1023-4e77-476b-9ecd-d81ca1769edf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ad6412a0adb4dfcb38eeed52caee583'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:24.710982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 909/1852] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:24 2022] DELETE /identity/v3/users/5ad6412a0adb4dfcb38eeed52caee583 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:24.715366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fab9b6-1bc5-4c90-b938-12f0f9d089f6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:24.715869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fab9b6-1bc5-4c90-b938-12f0f9d089f6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:24.720086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fab9b6-1bc5-4c90-b938-12f0f9d089f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:24.748621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d8fab9b6-1bc5-4c90-b938-12f0f9d089f6 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:24.749928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d8fab9b6-1bc5-4c90-b938-12f0f9d089f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:24.812415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 944/1853] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:24.823472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f1023-4e77-476b-9ecd-d81ca1769edf admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:24.824919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0e4f1023-4e77-476b-9ecd-d81ca1769edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:24.867365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:24.927916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f1023-4e77-476b-9ecd-d81ca1769edf admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:24.932018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13dc48-40c5-46f7-8904-1e3f1d2022da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:24.935240 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13dc48-40c5-46f7-8904-1e3f1d2022da service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:24.935240 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13dc48-40c5-46f7-8904-1e3f1d2022da service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:24.935240 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13dc48-40c5-46f7-8904-1e3f1d2022da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:24.945222 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13dc48-40c5-46f7-8904-1e3f1d2022da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:24.946700 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13dc48-40c5-46f7-8904-1e3f1d2022da service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:24.959280 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 910/1854] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:25.144080 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fab9b6-1bc5-4c90-b938-12f0f9d089f6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:25.146403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d8fab9b6-1bc5-4c90-b938-12f0f9d089f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:25.162253 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fab9b6-1bc5-4c90-b938-12f0f9d089f6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:25.167093 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:25.167727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:25.171208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:25.171208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] PATH_INFO: `/v3/projects/4ac36c09b5da4d0aa37782837e755bd6` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:25.171208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ac36c09b5da4d0aa37782837e755bd6)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:25.171757 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:25.330316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ac36c09b5da4d0aa37782837e755bd6', 'request_id': 'req-b01ae88f-e55a-4c39-9e39-325599d4dcb9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:25.426400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ac36c09b5da4d0aa37782837e755bd6', 'request_id': 'req-b01ae88f-e55a-4c39-9e39-325599d4dcb9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:25.438445 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ac36c09b5da4d0aa37782837e755bd6', 'request_id': 'req-b01ae88f-e55a-4c39-9e39-325599d4dcb9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:25.465124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] The token cache is being invalidate because project 4ac36c09b5da4d0aa37782837e755bd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:25.466340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:25.467839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:25.472233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 945/1855] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:25 2022] DELETE /identity/v3/projects/4ac36c09b5da4d0aa37782837e755bd6 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:25.476561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13dc48-40c5-46f7-8904-1e3f1d2022da service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:25.492773 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13dc48-40c5-46f7-8904-1e3f1d2022da service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:25.570952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f6510-a994-4c58-b042-e336b3f9d721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:25.571595 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f6510-a994-4c58-b042-e336b3f9d721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:25.571888 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f6510-a994-4c58-b042-e336b3f9d721 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:25.572170 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f6510-a994-4c58-b042-e336b3f9d721 admin admin] PATH_INFO: `/v3/users/fede191a176f4fa1b0d660345e094408` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:25.573339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f6510-a994-4c58-b042-e336b3f9d721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fede191a176f4fa1b0d660345e094408)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:25.575470 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0f6510-a994-4c58-b042-e336b3f9d721 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:25.588070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:25.589949 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:25.605747 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ae88f-e55a-4c39-9e39-325599d4dcb9 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:25.610962 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ae5a6e-1bb0-45a4-8304-7d65f02a060a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:25.611652 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae5a6e-1bb0-45a4-8304-7d65f02a060a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:25.611929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae5a6e-1bb0-45a4-8304-7d65f02a060a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:25.612250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ae5a6e-1bb0-45a4-8304-7d65f02a060a admin admin] PATH_INFO: `/v3/users/9bef888a851a4394aee4ea2987fc1b04` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:25.613583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae5a6e-1bb0-45a4-8304-7d65f02a060a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bef888a851a4394aee4ea2987fc1b04)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:25.615626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ae5a6e-1bb0-45a4-8304-7d65f02a060a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:25.795568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-bd0f6510-a994-4c58-b042-e336b3f9d721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fede191a176f4fa1b0d660345e094408'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:25.824488 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-bd0f6510-a994-4c58-b042-e336b3f9d721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fede191a176f4fa1b0d660345e094408'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:25.838073 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-74ae5a6e-1bb0-45a4-8304-7d65f02a060a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bef888a851a4394aee4ea2987fc1b04'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:25.911024 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-bd0f6510-a994-4c58-b042-e336b3f9d721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fede191a176f4fa1b0d660345e094408'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:25.911719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-bd0f6510-a994-4c58-b042-e336b3f9d721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fede191a176f4fa1b0d660345e094408'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:25.922158 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-bd0f6510-a994-4c58-b042-e336b3f9d721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fede191a176f4fa1b0d660345e094408'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:25.932864 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 911/1856] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:25 2022] DELETE /identity/v3/users/fede191a176f4fa1b0d660345e094408 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:25.935649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f6510-a994-4c58-b042-e336b3f9d721 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:25.942528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bd0f6510-a994-4c58-b042-e336b3f9d721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:26.007847 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-74ae5a6e-1bb0-45a4-8304-7d65f02a060a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bef888a851a4394aee4ea2987fc1b04'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:26.056072 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-74ae5a6e-1bb0-45a4-8304-7d65f02a060a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bef888a851a4394aee4ea2987fc1b04'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:26.058210 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-74ae5a6e-1bb0-45a4-8304-7d65f02a060a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bef888a851a4394aee4ea2987fc1b04'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:26.064399 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-74ae5a6e-1bb0-45a4-8304-7d65f02a060a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bef888a851a4394aee4ea2987fc1b04'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:26.103816 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 946/1857] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:25 2022] DELETE /identity/v3/users/9bef888a851a4394aee4ea2987fc1b04 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:26.109102 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3842a-0a4f-4801-be22-0f4320187b19 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:26.109388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3842a-0a4f-4801-be22-0f4320187b19 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:26.109692 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3842a-0a4f-4801-be22-0f4320187b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:26.124771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f6510-a994-4c58-b042-e336b3f9d721 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:26.132942 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7cc3842a-0a4f-4801-be22-0f4320187b19 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:26.134672 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7cc3842a-0a4f-4801-be22-0f4320187b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:26.212820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:26.213592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:26.214103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:26.214477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] PATH_INFO: `/v3/projects/0c46eccf65b5477c9a388c9b5644d819` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:26.216435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c46eccf65b5477c9a388c9b5644d819)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:26.218448 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:26.237863 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 947/1858] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:26.248563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3842a-0a4f-4801-be22-0f4320187b19 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:26.250850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7cc3842a-0a4f-4801-be22-0f4320187b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:26.320443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3842a-0a4f-4801-be22-0f4320187b19 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:26.330450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dae5b-ff45-4f22-a6cc-01464fd67ef7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:26.331085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dae5b-ff45-4f22-a6cc-01464fd67ef7 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:26.331793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dae5b-ff45-4f22-a6cc-01464fd67ef7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:26.332110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7dae5b-ff45-4f22-a6cc-01464fd67ef7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:26.351767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7dae5b-ff45-4f22-a6cc-01464fd67ef7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:26.352018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7dae5b-ff45-4f22-a6cc-01464fd67ef7 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:26.377276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 948/1859] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:26.377276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:26.394506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c46eccf65b5477c9a388c9b5644d819', 'request_id': 'req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:26.401815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c46eccf65b5477c9a388c9b5644d819', 'request_id': 'req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:26.410501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c46eccf65b5477c9a388c9b5644d819', 'request_id': 'req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:26.550921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] The token cache is being invalidate because project 0c46eccf65b5477c9a388c9b5644d819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:26.551215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:26.551927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:26.553594 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 912/1860] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:25 2022] DELETE /identity/v3/projects/0c46eccf65b5477c9a388c9b5644d819 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:26.557682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dae5b-ff45-4f22-a6cc-01464fd67ef7 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:26.559205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-be7dae5b-ff45-4f22-a6cc-01464fd67ef7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:26.572541 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7dae5b-ff45-4f22-a6cc-01464fd67ef7 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:26.629340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d011515-8e3a-463c-bfca-4d3bd8b25f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:26.629981 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d011515-8e3a-463c-bfca-4d3bd8b25f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:26.630262 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d011515-8e3a-463c-bfca-4d3bd8b25f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:26.630537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d011515-8e3a-463c-bfca-4d3bd8b25f89 admin admin] PATH_INFO: `/v3/users/fbb3cb80af9a4bbb99a810c1b2be3f8f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:26.631661 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d011515-8e3a-463c-bfca-4d3bd8b25f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbb3cb80af9a4bbb99a810c1b2be3f8f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:26.633949 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d011515-8e3a-463c-bfca-4d3bd8b25f89 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:26.710256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:26.711866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:26.725492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e5b73-a9ce-46e1-9e23-eb5e5556fe50 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:26.730077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:26.730692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:26.730966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:26.731231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2 admin admin] PATH_INFO: `/v3/projects/d378169677094294a58756721b74487c` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:26.732322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d378169677094294a58756721b74487c)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:26.734259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:26.740008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:26.882190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9d011515-8e3a-463c-bfca-4d3bd8b25f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbb3cb80af9a4bbb99a810c1b2be3f8f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:26.891799 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd378169677094294a58756721b74487c', 'request_id': 'req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:26.897294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd378169677094294a58756721b74487c', 'request_id': 'req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:26.903531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd378169677094294a58756721b74487c', 'request_id': 'req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:27.071489 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9d011515-8e3a-463c-bfca-4d3bd8b25f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbb3cb80af9a4bbb99a810c1b2be3f8f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:27.090621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9d011515-8e3a-463c-bfca-4d3bd8b25f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbb3cb80af9a4bbb99a810c1b2be3f8f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:27.093378 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9d011515-8e3a-463c-bfca-4d3bd8b25f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbb3cb80af9a4bbb99a810c1b2be3f8f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:27.094238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9d011515-8e3a-463c-bfca-4d3bd8b25f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbb3cb80af9a4bbb99a810c1b2be3f8f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:27.137153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 949/1861] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:26 2022] DELETE /identity/v3/users/fbb3cb80af9a4bbb99a810c1b2be3f8f => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:27.137153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d011515-8e3a-463c-bfca-4d3bd8b25f89 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:27.142537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9d011515-8e3a-463c-bfca-4d3bd8b25f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:27.192400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2 admin admin] The token cache is being invalidate because project d378169677094294a58756721b74487c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:27.192686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:27.194225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-903244c7-6ca7-4860-8fe7-d6fe8f0166b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:27.195899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 913/1862] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:26 2022] DELETE /identity/v3/projects/d378169677094294a58756721b74487c => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:27.202000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ca877-5a97-480e-b3f6-031b6b21fb06 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:27.202348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ca877-5a97-480e-b3f6-031b6b21fb06 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:27.202577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ca877-5a97-480e-b3f6-031b6b21fb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:27.225683 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-db0ca877-5a97-480e-b3f6-031b6b21fb06 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:27.226981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-db0ca877-5a97-480e-b3f6-031b6b21fb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:27.235084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 914/1863] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:27.242978 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64affc4f-737d-44ea-800f-8159891a99d2 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:27.243264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64affc4f-737d-44ea-800f-8159891a99d2 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:27.243543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64affc4f-737d-44ea-800f-8159891a99d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:27.245391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d011515-8e3a-463c-bfca-4d3bd8b25f89 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:27.250903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:27.251837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:27.252271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:27.252554 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] PATH_INFO: `/v3/projects/1c91dcee759c4ca382274395975ede91` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:27.253739 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c91dcee759c4ca382274395975ede91)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:27.255839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:27.265199 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-64affc4f-737d-44ea-800f-8159891a99d2 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:27.266469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-64affc4f-737d-44ea-800f-8159891a99d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:27.275794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 915/1864] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:27.279831 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64affc4f-737d-44ea-800f-8159891a99d2 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:27.293370 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64affc4f-737d-44ea-800f-8159891a99d2 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:27.301538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd57dc8-aced-4f57-a03c-8f73d636c83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:27.301538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd57dc8-aced-4f57-a03c-8f73d636c83f admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:27.301538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd57dc8-aced-4f57-a03c-8f73d636c83f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:27.301538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd57dc8-aced-4f57-a03c-8f73d636c83f admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:27.302488 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd57dc8-aced-4f57-a03c-8f73d636c83f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:27.302488 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd57dc8-aced-4f57-a03c-8f73d636c83f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:27.304741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd57dc8-aced-4f57-a03c-8f73d636c83f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:27.334795 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 916/1865] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:56:27.340325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd57dc8-aced-4f57-a03c-8f73d636c83f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:27.352457 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd57dc8-aced-4f57-a03c-8f73d636c83f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:27.357206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050c5ce-8db0-4b69-8d3c-b33b723d8689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:27.357823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050c5ce-8db0-4b69-8d3c-b33b723d8689 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:27.358124 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050c5ce-8db0-4b69-8d3c-b33b723d8689 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:27.358401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050c5ce-8db0-4b69-8d3c-b33b723d8689 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:27.359027 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050c5ce-8db0-4b69-8d3c-b33b723d8689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:27.361174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050c5ce-8db0-4b69-8d3c-b33b723d8689 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:27.476188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c91dcee759c4ca382274395975ede91', 'request_id': 'req-278ff2c2-b07c-43fd-948c-39db88a4a9e1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:27.611970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c91dcee759c4ca382274395975ede91', 'request_id': 'req-278ff2c2-b07c-43fd-948c-39db88a4a9e1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:27.622943 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 917/1866] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:27 2022] POST /identity/v3/projects => generated 377 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:27.625022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c91dcee759c4ca382274395975ede91', 'request_id': 'req-278ff2c2-b07c-43fd-948c-39db88a4a9e1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:27.629349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050c5ce-8db0-4b69-8d3c-b33b723d8689 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:27.655027 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] The token cache is being invalidate because project 1c91dcee759c4ca382274395975ede91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:27.655583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:27.657462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:27.670112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 950/1867] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:27 2022] DELETE /identity/v3/projects/1c91dcee759c4ca382274395975ede91 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:27.675091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:27.679300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:27.719777 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050c5ce-8db0-4b69-8d3c-b33b723d8689 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:27.727377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67905-bdd5-4666-b7c5-36e2e01f128a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:27.728516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67905-bdd5-4666-b7c5-36e2e01f128a admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:27.729009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67905-bdd5-4666-b7c5-36e2e01f128a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:27.730123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b67905-bdd5-4666-b7c5-36e2e01f128a admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:27.731645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b67905-bdd5-4666-b7c5-36e2e01f128a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:27.734485 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b67905-bdd5-4666-b7c5-36e2e01f128a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:27.775254 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ff2c2-b07c-43fd-948c-39db88a4a9e1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:27.791661 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4182165-67db-4763-82f7-1fcf3ae44eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:27.792354 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4182165-67db-4763-82f7-1fcf3ae44eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:27.793784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4182165-67db-4763-82f7-1fcf3ae44eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:27.795273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4182165-67db-4763-82f7-1fcf3ae44eb3 admin admin] PATH_INFO: `/v3/users/6a3524d90de743dbaed6df366039d05f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:27.796530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4182165-67db-4763-82f7-1fcf3ae44eb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a3524d90de743dbaed6df366039d05f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:27.799066 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4182165-67db-4763-82f7-1fcf3ae44eb3 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:27.855814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 918/1868] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:27 2022] POST /identity/v3/users => generated 289 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:27.863257 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67905-bdd5-4666-b7c5-36e2e01f128a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:27.868213 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-60b67905-bdd5-4666-b7c5-36e2e01f128a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:27.893979 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b67905-bdd5-4666-b7c5-36e2e01f128a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:27.899668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87e5f3-e8d5-4f07-9f47-aa6dd6344223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:27.900551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e5f3-e8d5-4f07-9f47-aa6dd6344223 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:27.900833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e5f3-e8d5-4f07-9f47-aa6dd6344223 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:27.901107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87e5f3-e8d5-4f07-9f47-aa6dd6344223 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:27.901825 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87e5f3-e8d5-4f07-9f47-aa6dd6344223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:27.904936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87e5f3-e8d5-4f07-9f47-aa6dd6344223 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:27.922140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 919/1869] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:27.929024 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87e5f3-e8d5-4f07-9f47-aa6dd6344223 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:27.948579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87e5f3-e8d5-4f07-9f47-aa6dd6344223 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:27.956113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dba46-355f-4cd8-9af6-01f932a6823c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:27.957093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dba46-355f-4cd8-9af6-01f932a6823c admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:27.957402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dba46-355f-4cd8-9af6-01f932a6823c admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:27.957678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7dba46-355f-4cd8-9af6-01f932a6823c admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:27.958388 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7dba46-355f-4cd8-9af6-01f932a6823c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:27.960572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7dba46-355f-4cd8-9af6-01f932a6823c admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:27.986087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 920/1870] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:27.986087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dba46-355f-4cd8-9af6-01f932a6823c admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:28.003307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7dba46-355f-4cd8-9af6-01f932a6823c admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:28.008719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c4182165-67db-4763-82f7-1fcf3ae44eb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a3524d90de743dbaed6df366039d05f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:28.009387 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac384c-4c27-489b-b01d-ab3ac5b1787d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:28.010137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac384c-4c27-489b-b01d-ab3ac5b1787d admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:28.011026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac384c-4c27-489b-b01d-ab3ac5b1787d admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:28.012927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac384c-4c27-489b-b01d-ab3ac5b1787d admin admin] PATH_INFO: `/v3/projects/4b300df70349459ca3134366ca97b134/users/088fc1d4dee84e6391c457c84d4506b9/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:28.030060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac384c-4c27-489b-b01d-ab3ac5b1787d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b300df70349459ca3134366ca97b134, user_id=088fc1d4dee84e6391c457c84d4506b9, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:28.032814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac384c-4c27-489b-b01d-ab3ac5b1787d admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:28.097888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c4182165-67db-4763-82f7-1fcf3ae44eb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a3524d90de743dbaed6df366039d05f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:28.109165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 921/1871] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:27 2022] PUT /identity/v3/projects/4b300df70349459ca3134366ca97b134/users/088fc1d4dee84e6391c457c84d4506b9/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:28.112037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac384c-4c27-489b-b01d-ab3ac5b1787d admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:28.114155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2aac384c-4c27-489b-b01d-ab3ac5b1787d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:28.129126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c4182165-67db-4763-82f7-1fcf3ae44eb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a3524d90de743dbaed6df366039d05f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:28.129126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c4182165-67db-4763-82f7-1fcf3ae44eb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a3524d90de743dbaed6df366039d05f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:28.129126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c4182165-67db-4763-82f7-1fcf3ae44eb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a3524d90de743dbaed6df366039d05f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:28.140439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 951/1872] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:27 2022] DELETE /identity/v3/users/6a3524d90de743dbaed6df366039d05f => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:28.146166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9eecce-01cb-4969-bc2d-a743d1678ff1 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:28.146525 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9eecce-01cb-4969-bc2d-a743d1678ff1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:28.146886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9eecce-01cb-4969-bc2d-a743d1678ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:28.203720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2d9eecce-01cb-4969-bc2d-a743d1678ff1 None None] MFA Rules not processed for user `088fc1d4dee84e6391c457c84d4506b9`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:28.205574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2d9eecce-01cb-4969-bc2d-a743d1678ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:28.240544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac384c-4c27-489b-b01d-ab3ac5b1787d admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:28.334500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 952/1873] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:28.349790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9eecce-01cb-4969-bc2d-a743d1678ff1 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:28.354825 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:28.355541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:28.355820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:28.356094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb admin admin] PATH_INFO: `/v3/projects/49d46c13d33347aaa1886999a5e88454` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:28.356529 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2d9eecce-01cb-4969-bc2d-a743d1678ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:28.357464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=49d46c13d33347aaa1886999a5e88454)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:28.359711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:28.474909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9eecce-01cb-4969-bc2d-a743d1678ff1 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:28.480603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092d6f4-ef6f-458e-81c3-b66f99035ce8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e021a00524341359bc4b5a800a9bbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:28.482145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092d6f4-ef6f-458e-81c3-b66f99035ce8 service glance] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:28.482501 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092d6f4-ef6f-458e-81c3-b66f99035ce8 service glance] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:28.482977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092d6f4-ef6f-458e-81c3-b66f99035ce8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:28.496604 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092d6f4-ef6f-458e-81c3-b66f99035ce8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:28.498177 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092d6f4-ef6f-458e-81c3-b66f99035ce8 service glance] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:28.510786 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 953/1874] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:28.564338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49d46c13d33347aaa1886999a5e88454', 'request_id': 'req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:28.572090 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49d46c13d33347aaa1886999a5e88454', 'request_id': 'req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:28.579197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49d46c13d33347aaa1886999a5e88454', 'request_id': 'req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:28.687976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb admin admin] The token cache is being invalidate because project 49d46c13d33347aaa1886999a5e88454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:28.689592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:28.690502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-60ebe7aa-874e-4fc2-9826-56f7ffaa63eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:28.692387 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 922/1875] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:28 2022] DELETE /identity/v3/projects/49d46c13d33347aaa1886999a5e88454 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:28.698506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8172d7-893b-41fd-bbd6-933fbeeff9da None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:28.698798 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8172d7-893b-41fd-bbd6-933fbeeff9da None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:28.699076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8172d7-893b-41fd-bbd6-933fbeeff9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:28.724505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bc8172d7-893b-41fd-bbd6-933fbeeff9da None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:28.726319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bc8172d7-893b-41fd-bbd6-933fbeeff9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:28.834625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 954/1876] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:28.852248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07ea9f-5c7e-4661-ac32-b02fd2f982d6 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:28.852580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07ea9f-5c7e-4661-ac32-b02fd2f982d6 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:28.853167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07ea9f-5c7e-4661-ac32-b02fd2f982d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:28.878848 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-0a07ea9f-5c7e-4661-ac32-b02fd2f982d6 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:28.880140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0a07ea9f-5c7e-4661-ac32-b02fd2f982d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:28.889600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 923/1877] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:28.894352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8172d7-893b-41fd-bbd6-933fbeeff9da None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:28.905393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8172d7-893b-41fd-bbd6-933fbeeff9da None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:28.910815 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0173da-417c-4f8f-820d-967963d4e508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:28.911459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0173da-417c-4f8f-820d-967963d4e508 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:28.911741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0173da-417c-4f8f-820d-967963d4e508 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:28.912016 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0173da-417c-4f8f-820d-967963d4e508 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:28.912661 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0173da-417c-4f8f-820d-967963d4e508 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:28.912956 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0173da-417c-4f8f-820d-967963d4e508 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:28.914908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0173da-417c-4f8f-820d-967963d4e508 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:28.929779 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 955/1878] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:56:28.933909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07ea9f-5c7e-4661-ac32-b02fd2f982d6 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:28.947075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07ea9f-5c7e-4661-ac32-b02fd2f982d6 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:28.952047 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e8265-5aba-43e1-b998-16146a487481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:28.952653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e8265-5aba-43e1-b998-16146a487481 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:28.953016 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e8265-5aba-43e1-b998-16146a487481 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:28.953315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e8265-5aba-43e1-b998-16146a487481 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:28.953999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e8265-5aba-43e1-b998-16146a487481 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:28.956582 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e8265-5aba-43e1-b998-16146a487481 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:29.002186 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0173da-417c-4f8f-820d-967963d4e508 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:29.003976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0c0173da-417c-4f8f-820d-967963d4e508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:29.019741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0173da-417c-4f8f-820d-967963d4e508 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:29.034781 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78472404-752a-44b9-816e-85ea624c0abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:29.035788 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78472404-752a-44b9-816e-85ea624c0abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:29.036261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78472404-752a-44b9-816e-85ea624c0abd admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:29.036681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78472404-752a-44b9-816e-85ea624c0abd admin admin] PATH_INFO: `/v3/projects/28ae8020a87e4b83a9f9b6a4e04e0eca` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:29.042679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78472404-752a-44b9-816e-85ea624c0abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=28ae8020a87e4b83a9f9b6a4e04e0eca)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:29.044951 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78472404-752a-44b9-816e-85ea624c0abd admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:29.112523 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 924/1879] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:28 2022] POST /identity/v3/projects => generated 359 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:29.116747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e8265-5aba-43e1-b998-16146a487481 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:29.191353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e8265-5aba-43e1-b998-16146a487481 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:29.202890 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca07183-8bd0-4e2c-a50a-02b39ea4986d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:29.202890 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca07183-8bd0-4e2c-a50a-02b39ea4986d admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:29.202890 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca07183-8bd0-4e2c-a50a-02b39ea4986d admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:29.202890 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca07183-8bd0-4e2c-a50a-02b39ea4986d admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:29.203335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca07183-8bd0-4e2c-a50a-02b39ea4986d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:29.203335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca07183-8bd0-4e2c-a50a-02b39ea4986d admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:29.246207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-78472404-752a-44b9-816e-85ea624c0abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28ae8020a87e4b83a9f9b6a4e04e0eca', 'request_id': 'req-78472404-752a-44b9-816e-85ea624c0abd'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:29.395035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 925/1880] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:29 2022] POST /identity/v3/users => generated 280 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:56:29.399267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca07183-8bd0-4e2c-a50a-02b39ea4986d admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:29.411694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca07183-8bd0-4e2c-a50a-02b39ea4986d admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:29.420043 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-78472404-752a-44b9-816e-85ea624c0abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28ae8020a87e4b83a9f9b6a4e04e0eca', 'request_id': 'req-78472404-752a-44b9-816e-85ea624c0abd'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:29.443251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-78472404-752a-44b9-816e-85ea624c0abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28ae8020a87e4b83a9f9b6a4e04e0eca', 'request_id': 'req-78472404-752a-44b9-816e-85ea624c0abd'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:29.455706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-78472404-752a-44b9-816e-85ea624c0abd admin admin] The token cache is being invalidate because project 28ae8020a87e4b83a9f9b6a4e04e0eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:29.456040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-78472404-752a-44b9-816e-85ea624c0abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:29.456805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-78472404-752a-44b9-816e-85ea624c0abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:29.458742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 956/1881] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:29 2022] DELETE /identity/v3/projects/28ae8020a87e4b83a9f9b6a4e04e0eca => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:29.463183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4071835-f6c6-4161-aa15-fc1ae17539c1 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:29.463468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4071835-f6c6-4161-aa15-fc1ae17539c1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:29.463741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4071835-f6c6-4161-aa15-fc1ae17539c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:29.485649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8028e8-7642-4323-b8ba-22153e80b9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:29.486291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8028e8-7642-4323-b8ba-22153e80b9ae admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:29.486581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8028e8-7642-4323-b8ba-22153e80b9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:29.486864 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8028e8-7642-4323-b8ba-22153e80b9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:29.487279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d4071835-f6c6-4161-aa15-fc1ae17539c1 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:29.487502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8028e8-7642-4323-b8ba-22153e80b9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:29.488914 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d4071835-f6c6-4161-aa15-fc1ae17539c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:29.489475 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8028e8-7642-4323-b8ba-22153e80b9ae admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:29.501742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 926/1883] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:56:29.506333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7a825-5314-45ea-98b0-9902e8b0cfed None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:29.506618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7a825-5314-45ea-98b0-9902e8b0cfed None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:29.506975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d7a825-5314-45ea-98b0-9902e8b0cfed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:29.508793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 957/1882] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:29.509772 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4071835-f6c6-4161-aa15-fc1ae17539c1 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:29.511307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d4071835-f6c6-4161-aa15-fc1ae17539c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:29.526564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4071835-f6c6-4161-aa15-fc1ae17539c1 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:29.536505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c192807-bf3b-46bb-a333-70ef68d6e176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:29.537152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c192807-bf3b-46bb-a333-70ef68d6e176 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:29.537429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c192807-bf3b-46bb-a333-70ef68d6e176 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:29.538737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-00d7a825-5314-45ea-98b0-9902e8b0cfed None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:29.540024 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-00d7a825-5314-45ea-98b0-9902e8b0cfed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:29.540939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c192807-bf3b-46bb-a333-70ef68d6e176 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:29.541601 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c192807-bf3b-46bb-a333-70ef68d6e176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:29.543248 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c192807-bf3b-46bb-a333-70ef68d6e176 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:29.559521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 958/1884] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:29.569285 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c192807-bf3b-46bb-a333-70ef68d6e176 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:29.573710 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 927/1885] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:29.577911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7a825-5314-45ea-98b0-9902e8b0cfed None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:29.597143 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d7a825-5314-45ea-98b0-9902e8b0cfed None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:29.623314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c192807-bf3b-46bb-a333-70ef68d6e176 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:29.632745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce88cad-79d0-4e46-ad67-4230ef7f072e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:29.633662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce88cad-79d0-4e46-ad67-4230ef7f072e admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:29.636702 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce88cad-79d0-4e46-ad67-4230ef7f072e admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:29.636990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce88cad-79d0-4e46-ad67-4230ef7f072e admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:29.638156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce88cad-79d0-4e46-ad67-4230ef7f072e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:29.638156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce88cad-79d0-4e46-ad67-4230ef7f072e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:29.639733 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce88cad-79d0-4e46-ad67-4230ef7f072e admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:29.648409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8267f61-22d5-4b40-8188-8289b87aa75e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:29.649100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8267f61-22d5-4b40-8188-8289b87aa75e admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:29.649384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8267f61-22d5-4b40-8188-8289b87aa75e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:29.649697 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8267f61-22d5-4b40-8188-8289b87aa75e admin admin] PATH_INFO: `/v3/projects/546b70f8cc3e40a983a12ce509d5f92e/users/b553b8a374234197a126b658ccde4bd4/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:29.673122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 928/1886] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:56:29.677026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8267f61-22d5-4b40-8188-8289b87aa75e admin admin] RBAC: Authorizing `identity:create_grant(project_id=546b70f8cc3e40a983a12ce509d5f92e, user_id=b553b8a374234197a126b658ccde4bd4, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:29.682832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce88cad-79d0-4e46-ad67-4230ef7f072e admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:29.683456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8267f61-22d5-4b40-8188-8289b87aa75e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:29.698717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce88cad-79d0-4e46-ad67-4230ef7f072e admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:29.707101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591160a7-6f5f-4083-a429-ac2fec55e5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:29.708026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591160a7-6f5f-4083-a429-ac2fec55e5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:29.708500 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591160a7-6f5f-4083-a429-ac2fec55e5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:29.708922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591160a7-6f5f-4083-a429-ac2fec55e5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:29.709704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591160a7-6f5f-4083-a429-ac2fec55e5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:29.712242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591160a7-6f5f-4083-a429-ac2fec55e5e7 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:29.788566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 959/1887] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:29 2022] PUT /identity/v3/projects/546b70f8cc3e40a983a12ce509d5f92e/users/b553b8a374234197a126b658ccde4bd4/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:29.799184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b9657-6bf0-4c24-a862-f7501cd1fec4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:29.800935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b9657-6bf0-4c24-a862-f7501cd1fec4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:29.801263 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250b9657-6bf0-4c24-a862-f7501cd1fec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:29.835137 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:29.845123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 929/1888] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:29 2022] POST /identity/v3/projects => generated 379 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:29.845298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-250b9657-6bf0-4c24-a862-f7501cd1fec4 None None] MFA Rules not processed for user `b553b8a374234197a126b658ccde4bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:29.845624 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-250b9657-6bf0-4c24-a862-f7501cd1fec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:29.846595 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591160a7-6f5f-4083-a429-ac2fec55e5e7 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:29.965513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 960/1889] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:29.986685 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591160a7-6f5f-4083-a429-ac2fec55e5e7 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:29.997841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a65fa-4707-4e2c-975e-2340e48053e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:29.997841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a65fa-4707-4e2c-975e-2340e48053e9 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:29.997841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a65fa-4707-4e2c-975e-2340e48053e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:29.997841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9a65fa-4707-4e2c-975e-2340e48053e9 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:30.009548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a65fa-4707-4e2c-975e-2340e48053e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:30.009548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9a65fa-4707-4e2c-975e-2340e48053e9 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:30.011857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b9657-6bf0-4c24-a862-f7501cd1fec4 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:30.014767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-250b9657-6bf0-4c24-a862-f7501cd1fec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:30.100619 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250b9657-6bf0-4c24-a862-f7501cd1fec4 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:30.104658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dad0c9-1391-4bb0-b568-39429a107286 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:30.105263 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dad0c9-1391-4bb0-b568-39429a107286 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:30.105536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dad0c9-1391-4bb0-b568-39429a107286 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:30.105829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dad0c9-1391-4bb0-b568-39429a107286 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:30.115168 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dad0c9-1391-4bb0-b568-39429a107286 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:30.116688 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dad0c9-1391-4bb0-b568-39429a107286 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:30.122095 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:30.128333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 961/1890] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:30 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:56:30.129838 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 930/1891] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:29 2022] POST /identity/v3/users => generated 290 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:30.133036 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dad0c9-1391-4bb0-b568-39429a107286 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:30.145600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dad0c9-1391-4bb0-b568-39429a107286 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:30.149149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a65fa-4707-4e2c-975e-2340e48053e9 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:30.154167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407aca97-3521-4081-9437-a7ff822817bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:30.155030 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407aca97-3521-4081-9437-a7ff822817bc admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:30.155437 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407aca97-3521-4081-9437-a7ff822817bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:30.155805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407aca97-3521-4081-9437-a7ff822817bc admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:30.156700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407aca97-3521-4081-9437-a7ff822817bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:30.159610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407aca97-3521-4081-9437-a7ff822817bc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:30.161433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9a65fa-4707-4e2c-975e-2340e48053e9 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:30.166216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8c7e5-c794-4c5d-9720-648610364162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:30.167404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8c7e5-c794-4c5d-9720-648610364162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:30.167928 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8c7e5-c794-4c5d-9720-648610364162 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:30.172581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8c7e5-c794-4c5d-9720-648610364162 admin admin] PATH_INFO: `/v3/users/b553b8a374234197a126b658ccde4bd4` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:30.172581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8c7e5-c794-4c5d-9720-648610364162 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b553b8a374234197a126b658ccde4bd4)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:30.172581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8c7e5-c794-4c5d-9720-648610364162 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:30.178102 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 962/1892] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:56:30.182144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407aca97-3521-4081-9437-a7ff822817bc admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:30.196255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407aca97-3521-4081-9437-a7ff822817bc admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:30.203342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48d05f-3e0f-4044-9614-11b0ae8a6f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:30.203342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48d05f-3e0f-4044-9614-11b0ae8a6f33 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:30.203342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48d05f-3e0f-4044-9614-11b0ae8a6f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:30.203342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48d05f-3e0f-4044-9614-11b0ae8a6f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:30.203635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48d05f-3e0f-4044-9614-11b0ae8a6f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:30.207222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48d05f-3e0f-4044-9614-11b0ae8a6f33 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:30.221318 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 963/1893] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:30.225673 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48d05f-3e0f-4044-9614-11b0ae8a6f33 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:30.241212 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48d05f-3e0f-4044-9614-11b0ae8a6f33 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:30.248203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7fc13-5a33-4853-b8b8-d2faec212ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:30.248922 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7fc13-5a33-4853-b8b8-d2faec212ea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:30.250904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7fc13-5a33-4853-b8b8-d2faec212ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:30.251183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7fc13-5a33-4853-b8b8-d2faec212ea9 admin admin] PATH_INFO: `/v3/projects/81b2e4b2b5a94284a76fa794e402b583/users/5f174f5a7732486bb307343d133df347/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:30.276459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7fc13-5a33-4853-b8b8-d2faec212ea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81b2e4b2b5a94284a76fa794e402b583, user_id=5f174f5a7732486bb307343d133df347, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:30.279614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7fc13-5a33-4853-b8b8-d2faec212ea9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:30.369462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 964/1894] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:30 2022] PUT /identity/v3/projects/81b2e4b2b5a94284a76fa794e402b583/users/5f174f5a7732486bb307343d133df347/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:30.384605 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca1067-036a-4272-8111-90b4cf8c7d61 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:30.385653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca1067-036a-4272-8111-90b4cf8c7d61 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:30.387007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca1067-036a-4272-8111-90b4cf8c7d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:30.399051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-aee8c7e5-c794-4c5d-9720-648610364162 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b553b8a374234197a126b658ccde4bd4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:30.421091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-aee8c7e5-c794-4c5d-9720-648610364162 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b553b8a374234197a126b658ccde4bd4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:30.441767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-70ca1067-036a-4272-8111-90b4cf8c7d61 None None] MFA Rules not processed for user `5f174f5a7732486bb307343d133df347`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:30.443105 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-70ca1067-036a-4272-8111-90b4cf8c7d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:30.523293 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 965/1895] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:30.527732 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca1067-036a-4272-8111-90b4cf8c7d61 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:30.563768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-aee8c7e5-c794-4c5d-9720-648610364162 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b553b8a374234197a126b658ccde4bd4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:30.570802 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-aee8c7e5-c794-4c5d-9720-648610364162 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b553b8a374234197a126b658ccde4bd4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:30.581714 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-aee8c7e5-c794-4c5d-9720-648610364162 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b553b8a374234197a126b658ccde4bd4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:30.589128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 931/1896] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:30 2022] DELETE /identity/v3/users/b553b8a374234197a126b658ccde4bd4 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:30.598929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147ad7c-92a1-489c-b8c1-52aa74df83bd None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:30.598929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147ad7c-92a1-489c-b8c1-52aa74df83bd None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:30.598929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4147ad7c-92a1-489c-b8c1-52aa74df83bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:30.600380 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca1067-036a-4272-8111-90b4cf8c7d61 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:30.617423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-4147ad7c-92a1-489c-b8c1-52aa74df83bd None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:30.618691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4147ad7c-92a1-489c-b8c1-52aa74df83bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:30.684331 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace880f4-bc7f-4a85-9274-9e12119381ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:30.713413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace880f4-bc7f-4a85-9274-9e12119381ab admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:30.713413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace880f4-bc7f-4a85-9274-9e12119381ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:30.713413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace880f4-bc7f-4a85-9274-9e12119381ab admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:30.713413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace880f4-bc7f-4a85-9274-9e12119381ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:30.713413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace880f4-bc7f-4a85-9274-9e12119381ab admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:30.714889 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 932/1897] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:30.725163 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147ad7c-92a1-489c-b8c1-52aa74df83bd None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:30.726841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4147ad7c-92a1-489c-b8c1-52aa74df83bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:30.828757 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4147ad7c-92a1-489c-b8c1-52aa74df83bd None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:30.834300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78980c7a-0029-4ac2-b240-b64c6cefe0e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:30.835052 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78980c7a-0029-4ac2-b240-b64c6cefe0e2 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:30.835434 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78980c7a-0029-4ac2-b240-b64c6cefe0e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:30.835817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78980c7a-0029-4ac2-b240-b64c6cefe0e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:30.851403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78980c7a-0029-4ac2-b240-b64c6cefe0e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:30.855180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78980c7a-0029-4ac2-b240-b64c6cefe0e2 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:30.877651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 933/1898] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:30.882115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 966/1899] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:30 2022] POST /identity/v3/projects => generated 381 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:56:30.896051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78980c7a-0029-4ac2-b240-b64c6cefe0e2 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:30.903169 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-78980c7a-0029-4ac2-b240-b64c6cefe0e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:30.922035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78980c7a-0029-4ac2-b240-b64c6cefe0e2 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:31.024260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142930a7-5386-453f-aea3-2147c1172d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:31.024970 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142930a7-5386-453f-aea3-2147c1172d43 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:31.025149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142930a7-5386-453f-aea3-2147c1172d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:31.025420 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142930a7-5386-453f-aea3-2147c1172d43 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:31.026138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142930a7-5386-453f-aea3-2147c1172d43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:31.028325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142930a7-5386-453f-aea3-2147c1172d43 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:31.168866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 934/1900] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:30 2022] POST /identity/v3/users => generated 297 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:31.183373 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace880f4-bc7f-4a85-9274-9e12119381ab admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:31.206030 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace880f4-bc7f-4a85-9274-9e12119381ab admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:31.212764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee934a-1ee9-4092-b87d-bf85ba7c2417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:31.213420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee934a-1ee9-4092-b87d-bf85ba7c2417 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:31.213703 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee934a-1ee9-4092-b87d-bf85ba7c2417 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:31.218255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee934a-1ee9-4092-b87d-bf85ba7c2417 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:31.219244 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee934a-1ee9-4092-b87d-bf85ba7c2417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:31.221966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ee934a-1ee9-4092-b87d-bf85ba7c2417 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:31.243893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 967/1901] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:31.245482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142930a7-5386-453f-aea3-2147c1172d43 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:31.257229 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142930a7-5386-453f-aea3-2147c1172d43 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:31.263581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b08c4d-a9c5-4761-ba73-12109a564729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:31.267256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b08c4d-a9c5-4761-ba73-12109a564729 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:31.267846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b08c4d-a9c5-4761-ba73-12109a564729 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:31.268184 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b08c4d-a9c5-4761-ba73-12109a564729 admin admin] PATH_INFO: `/v3/projects/ec55bec566904ad083decc70ee7003e5/users/e07851467d274f5c8f8e3cec27719c49/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:31.285997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b08c4d-a9c5-4761-ba73-12109a564729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec55bec566904ad083decc70ee7003e5, user_id=e07851467d274f5c8f8e3cec27719c49, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:31.289674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b08c4d-a9c5-4761-ba73-12109a564729 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:31.438558 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 935/1902] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:31 2022] PUT /identity/v3/projects/ec55bec566904ad083decc70ee7003e5/users/e07851467d274f5c8f8e3cec27719c49/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:31.442518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee934a-1ee9-4092-b87d-bf85ba7c2417 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:31.460349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee934a-1ee9-4092-b87d-bf85ba7c2417 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:31.551112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677b25d-d36b-458b-9622-fb123af8e0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:31.552202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677b25d-d36b-458b-9622-fb123af8e0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:31.552525 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677b25d-d36b-458b-9622-fb123af8e0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:31.552959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677b25d-d36b-458b-9622-fb123af8e0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:31.554032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677b25d-d36b-458b-9622-fb123af8e0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:31.555936 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677b25d-d36b-458b-9622-fb123af8e0f4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:31.575006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 968/1903] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:31.579069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b08c4d-a9c5-4761-ba73-12109a564729 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:31.593752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b08c4d-a9c5-4761-ba73-12109a564729 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:31.600728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bada88-fbe9-45e1-b736-c04118218dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:31.601654 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bada88-fbe9-45e1-b736-c04118218dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:31.609493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bada88-fbe9-45e1-b736-c04118218dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:31.609493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bada88-fbe9-45e1-b736-c04118218dc0 admin admin] PATH_INFO: `/v3/projects/ec55bec566904ad083decc70ee7003e5/users/e07851467d274f5c8f8e3cec27719c49/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:31.609493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bada88-fbe9-45e1-b736-c04118218dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec55bec566904ad083decc70ee7003e5, user_id=e07851467d274f5c8f8e3cec27719c49, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:31.609493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bada88-fbe9-45e1-b736-c04118218dc0 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:31.622875 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 936/1904] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:31 2022] PUT /identity/v3/projects/ec55bec566904ad083decc70ee7003e5/users/e07851467d274f5c8f8e3cec27719c49/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:31.634205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea2fa4-199b-4972-943d-758814387320 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:31.634777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea2fa4-199b-4972-943d-758814387320 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:31.635110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ea2fa4-199b-4972-943d-758814387320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:31.677179 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bada88-fbe9-45e1-b736-c04118218dc0 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:31.678000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-17ea2fa4-199b-4972-943d-758814387320 None None] MFA Rules not processed for user `e07851467d274f5c8f8e3cec27719c49`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:31.679268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-17ea2fa4-199b-4972-943d-758814387320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:31.679648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-60bada88-fbe9-45e1-b736-c04118218dc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:31.824442 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bada88-fbe9-45e1-b736-c04118218dc0 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:31.866564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 969/1905] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:31 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:31.871060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:31.872457 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:31.873013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:31.873471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df admin admin] PATH_INFO: `/v3/projects/546b70f8cc3e40a983a12ce509d5f92e` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:31.875365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df admin admin] RBAC: Authorizing `identity:delete_project(project_id=546b70f8cc3e40a983a12ce509d5f92e)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:31.878819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:31.881330 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea2fa4-199b-4972-943d-758814387320 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:31.883092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-17ea2fa4-199b-4972-943d-758814387320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:31.950480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ea2fa4-199b-4972-943d-758814387320 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:31.955976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4c8c7-5fcb-471d-b807-2c4124d153ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:31.973156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4c8c7-5fcb-471d-b807-2c4124d153ad service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:31.973156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4c8c7-5fcb-471d-b807-2c4124d153ad service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:31.973156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4c8c7-5fcb-471d-b807-2c4124d153ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:31.994070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4c8c7-5fcb-471d-b807-2c4124d153ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:31.995753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4c8c7-5fcb-471d-b807-2c4124d153ad service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:32.024116 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 970/1906] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:56:32.025361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '546b70f8cc3e40a983a12ce509d5f92e', 'request_id': 'req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:32.035727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '546b70f8cc3e40a983a12ce509d5f92e', 'request_id': 'req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:32.043857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '546b70f8cc3e40a983a12ce509d5f92e', 'request_id': 'req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:32.156020 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df admin admin] The token cache is being invalidate because project 546b70f8cc3e40a983a12ce509d5f92e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:32.156356 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:32.156973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-95d5f50a-ceb3-4b88-996b-d9e2ed7ac3df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:32.158796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 937/1907] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:31 2022] DELETE /identity/v3/projects/546b70f8cc3e40a983a12ce509d5f92e => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:32.163542 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb525c82-41cb-45d5-b373-31f79a5902ed None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:32.163908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb525c82-41cb-45d5-b373-31f79a5902ed None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:32.164275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb525c82-41cb-45d5-b373-31f79a5902ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:32.189789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bb525c82-41cb-45d5-b373-31f79a5902ed None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:32.191121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bb525c82-41cb-45d5-b373-31f79a5902ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:32.290499 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 971/1908] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:32.301198 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec90f9-db6e-4d9d-ab36-b9f226f23fc9 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:32.301613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec90f9-db6e-4d9d-ab36-b9f226f23fc9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:32.301872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec90f9-db6e-4d9d-ab36-b9f226f23fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:32.316842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:32.334936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-c7ec90f9-db6e-4d9d-ab36-b9f226f23fc9 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:32.336727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c7ec90f9-db6e-4d9d-ab36-b9f226f23fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:32.359935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 938/1909] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:32.364735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb525c82-41cb-45d5-b373-31f79a5902ed None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:32.376970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb525c82-41cb-45d5-b373-31f79a5902ed None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:32.383215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0603ba-41c0-47fa-bfb5-4dc4dff3fcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:32.387054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0603ba-41c0-47fa-bfb5-4dc4dff3fcde admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:32.387389 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0603ba-41c0-47fa-bfb5-4dc4dff3fcde admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:32.387668 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0603ba-41c0-47fa-bfb5-4dc4dff3fcde admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:32.388426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0603ba-41c0-47fa-bfb5-4dc4dff3fcde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:32.388829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0603ba-41c0-47fa-bfb5-4dc4dff3fcde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:32.390578 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0603ba-41c0-47fa-bfb5-4dc4dff3fcde admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:32.408603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 972/1910] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:56:32.424758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec90f9-db6e-4d9d-ab36-b9f226f23fc9 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:32.449573 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec90f9-db6e-4d9d-ab36-b9f226f23fc9 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:32.459178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94738ed-8266-45d5-82c9-7bf3a654023a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:32.459178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94738ed-8266-45d5-82c9-7bf3a654023a admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:32.459178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94738ed-8266-45d5-82c9-7bf3a654023a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:32.459178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94738ed-8266-45d5-82c9-7bf3a654023a admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:32.459653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94738ed-8266-45d5-82c9-7bf3a654023a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:32.460327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94738ed-8266-45d5-82c9-7bf3a654023a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:32.594993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 939/1911] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:32 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:32.599672 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0603ba-41c0-47fa-bfb5-4dc4dff3fcde admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:32.611150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:32.613681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0603ba-41c0-47fa-bfb5-4dc4dff3fcde admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:32.677240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9606fa6-fd43-49f8-b820-8f125e9f593f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:32.677919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9606fa6-fd43-49f8-b820-8f125e9f593f admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:32.678223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9606fa6-fd43-49f8-b820-8f125e9f593f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:32.678509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9606fa6-fd43-49f8-b820-8f125e9f593f admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:32.679161 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9606fa6-fd43-49f8-b820-8f125e9f593f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:32.688299 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9606fa6-fd43-49f8-b820-8f125e9f593f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:32.816739 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 973/1912] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:32 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:56:32.818284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94738ed-8266-45d5-82c9-7bf3a654023a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:32.828766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94738ed-8266-45d5-82c9-7bf3a654023a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:32.834253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4d9c1-123e-427c-82d4-d6fd119e1171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:32.835113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4d9c1-123e-427c-82d4-d6fd119e1171 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:32.835383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4d9c1-123e-427c-82d4-d6fd119e1171 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:32.835660 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4d9c1-123e-427c-82d4-d6fd119e1171 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:32.836367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4d9c1-123e-427c-82d4-d6fd119e1171 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:32.838115 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4d9c1-123e-427c-82d4-d6fd119e1171 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:32.853039 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 940/1913] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:32.854600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9606fa6-fd43-49f8-b820-8f125e9f593f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:32.871611 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9606fa6-fd43-49f8-b820-8f125e9f593f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:32.873549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc19687-7ed9-47eb-bd2d-a0edd799a97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:32.874232 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc19687-7ed9-47eb-bd2d-a0edd799a97f admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:32.874455 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc19687-7ed9-47eb-bd2d-a0edd799a97f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:32.874724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc19687-7ed9-47eb-bd2d-a0edd799a97f admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:32.875558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc19687-7ed9-47eb-bd2d-a0edd799a97f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:32.877544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc19687-7ed9-47eb-bd2d-a0edd799a97f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:32.891064 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 974/1914] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:32.895664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4d9c1-123e-427c-82d4-d6fd119e1171 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:32.908796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4d9c1-123e-427c-82d4-d6fd119e1171 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:32.914309 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81db753-beaf-4437-8810-3b875d6c4e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:32.915106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81db753-beaf-4437-8810-3b875d6c4e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:32.915911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81db753-beaf-4437-8810-3b875d6c4e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:32.915911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81db753-beaf-4437-8810-3b875d6c4e84 admin admin] PATH_INFO: `/v3/projects/11fae649642b4703b649d02268bdb4eb/users/0ebafc08399c4853959800883742eb78/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:32.934902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81db753-beaf-4437-8810-3b875d6c4e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fae649642b4703b649d02268bdb4eb, user_id=0ebafc08399c4853959800883742eb78, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:32.937757 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81db753-beaf-4437-8810-3b875d6c4e84 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:33.076467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 941/1915] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:32 2022] PUT /identity/v3/projects/11fae649642b4703b649d02268bdb4eb/users/0ebafc08399c4853959800883742eb78/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:33.082096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159bc4e-c3cb-451c-9d17-6cceb1756c08 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:33.082426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159bc4e-c3cb-451c-9d17-6cceb1756c08 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:33.082777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9159bc4e-c3cb-451c-9d17-6cceb1756c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:33.114648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-9159bc4e-c3cb-451c-9d17-6cceb1756c08 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:33.116565 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9159bc4e-c3cb-451c-9d17-6cceb1756c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:33.217116 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 975/1916] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:33.245991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81db753-beaf-4437-8810-3b875d6c4e84 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:33.248003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b81db753-beaf-4437-8810-3b875d6c4e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:33.333514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159bc4e-c3cb-451c-9d17-6cceb1756c08 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:33.335090 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9159bc4e-c3cb-451c-9d17-6cceb1756c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:33.392458 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81db753-beaf-4437-8810-3b875d6c4e84 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:33.399102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b23f85-07bf-45a7-81d8-cd3c51ebd46b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:33.399962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b23f85-07bf-45a7-81d8-cd3c51ebd46b service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:33.400243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b23f85-07bf-45a7-81d8-cd3c51ebd46b service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:33.400518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b23f85-07bf-45a7-81d8-cd3c51ebd46b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:33.419047 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b23f85-07bf-45a7-81d8-cd3c51ebd46b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:33.420538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b23f85-07bf-45a7-81d8-cd3c51ebd46b service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:33.433598 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 942/1917] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:33.468389 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9159bc4e-c3cb-451c-9d17-6cceb1756c08 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:33.472036 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06999b0-1f7b-4b98-860a-31d02f6649f0 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07851467d274f5c8f8e3cec27719c49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec55bec566904ad083decc70ee7003e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:33.472782 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06999b0-1f7b-4b98-860a-31d02f6649f0 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:33.473170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06999b0-1f7b-4b98-860a-31d02f6649f0 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:33.473951 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06999b0-1f7b-4b98-860a-31d02f6649f0 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:33.475242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06999b0-1f7b-4b98-860a-31d02f6649f0 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:33.475674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06999b0-1f7b-4b98-860a-31d02f6649f0 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:33.477744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06999b0-1f7b-4b98-860a-31d02f6649f0 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:33.497841 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 976/1918] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:56:33.502283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b23f85-07bf-45a7-81d8-cd3c51ebd46b service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:33.514837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b23f85-07bf-45a7-81d8-cd3c51ebd46b service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:33.523945 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b17ff-f56b-425a-9910-89cec5267cf3 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07851467d274f5c8f8e3cec27719c49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec55bec566904ad083decc70ee7003e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:33.524865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b17ff-f56b-425a-9910-89cec5267cf3 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:33.525330 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b17ff-f56b-425a-9910-89cec5267cf3 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:33.525980 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b17ff-f56b-425a-9910-89cec5267cf3 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:33.526802 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b17ff-f56b-425a-9910-89cec5267cf3 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:33.528924 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660b17ff-f56b-425a-9910-89cec5267cf3 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:33.694387 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 943/1919] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:33 2022] POST /identity/v3/projects => generated 354 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:33.697009 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06999b0-1f7b-4b98-860a-31d02f6649f0 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:33.708219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06999b0-1f7b-4b98-860a-31d02f6649f0 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:33.776174 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d5d85-d339-485c-9c2d-63ab04faa5d8 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07851467d274f5c8f8e3cec27719c49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec55bec566904ad083decc70ee7003e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:33.776796 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d5d85-d339-485c-9c2d-63ab04faa5d8 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:33.777078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d5d85-d339-485c-9c2d-63ab04faa5d8 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:33.777403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d5d85-d339-485c-9c2d-63ab04faa5d8 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:33.778319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d5d85-d339-485c-9c2d-63ab04faa5d8 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:33.778623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d5d85-d339-485c-9c2d-63ab04faa5d8 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:33.780585 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d5d85-d339-485c-9c2d-63ab04faa5d8 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:33.794501 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 977/1920] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:56:34.067584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b17ff-f56b-425a-9910-89cec5267cf3 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:34.084095 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b17ff-f56b-425a-9910-89cec5267cf3 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:34.091177 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e07851467d274f5c8f8e3cec27719c49', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ec55bec566904ad083decc70ee7003e5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:34.091925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:34.092416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:34.092827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] PATH_INFO: `/v3/projects/7a03f214a19745a1aeddd3197a950c1f` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:34.094197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7a03f214a19745a1aeddd3197a950c1f)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:34.096929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:34.215168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a03f214a19745a1aeddd3197a950c1f', 'request_id': 'req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:34.221676 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a03f214a19745a1aeddd3197a950c1f', 'request_id': 'req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:34.233474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a03f214a19745a1aeddd3197a950c1f', 'request_id': 'req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:34.350837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] The token cache is being invalidate because project 7a03f214a19745a1aeddd3197a950c1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:34.351395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:34.352027 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d0d0da33-b8f1-40ce-a4cd-0d2470354b78 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:34.353986 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 944/1921] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:34 2022] DELETE /identity/v3/projects/7a03f214a19745a1aeddd3197a950c1f => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:34.455665 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:35.069132 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:35.157205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d5d85-d339-485c-9c2d-63ab04faa5d8 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:35.158889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-010d5d85-d339-485c-9c2d-63ab04faa5d8 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:35.252139 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d5d85-d339-485c-9c2d-63ab04faa5d8 tempest-VolumeQuotaClassesTest-1046605692 tempest-VolumeQuotaClassesTest-1046605692-project-admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:35.311157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5b5df-cfd8-4030-8ca8-ba0c5ba41412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:35.311814 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5b5df-cfd8-4030-8ca8-ba0c5ba41412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:35.314695 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5b5df-cfd8-4030-8ca8-ba0c5ba41412 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:35.314695 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b5b5df-cfd8-4030-8ca8-ba0c5ba41412 admin admin] PATH_INFO: `/v3/users/5f174f5a7732486bb307343d133df347` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:35.314695 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b5b5df-cfd8-4030-8ca8-ba0c5ba41412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f174f5a7732486bb307343d133df347)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:35.316075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b5b5df-cfd8-4030-8ca8-ba0c5ba41412 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:35.609587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7b5b5df-cfd8-4030-8ca8-ba0c5ba41412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f174f5a7732486bb307343d133df347'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:35.693404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7b5b5df-cfd8-4030-8ca8-ba0c5ba41412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f174f5a7732486bb307343d133df347'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:35.708449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7b5b5df-cfd8-4030-8ca8-ba0c5ba41412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f174f5a7732486bb307343d133df347'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:35.709737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7b5b5df-cfd8-4030-8ca8-ba0c5ba41412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f174f5a7732486bb307343d133df347'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:35.710768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7b5b5df-cfd8-4030-8ca8-ba0c5ba41412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f174f5a7732486bb307343d133df347'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:35.731597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 978/1922] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:35 2022] DELETE /identity/v3/users/5f174f5a7732486bb307343d133df347 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:35.735863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59b8d5-ebd5-4486-8e94-568b1b666cd7 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:35.736869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59b8d5-ebd5-4486-8e94-568b1b666cd7 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:35.737293 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c59b8d5-ebd5-4486-8e94-568b1b666cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:35.758170 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-6c59b8d5-ebd5-4486-8e94-568b1b666cd7 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:35.759219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6c59b8d5-ebd5-4486-8e94-568b1b666cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:35.833362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 945/1923] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:35.843201 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5b5df-cfd8-4030-8ca8-ba0c5ba41412 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:35.844608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d7b5b5df-cfd8-4030-8ca8-ba0c5ba41412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:35.916198 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b5b5df-cfd8-4030-8ca8-ba0c5ba41412 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:35.920756 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcafeb-7e80-4c1e-b1a7-0eb583f134ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:35.921579 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcafeb-7e80-4c1e-b1a7-0eb583f134ac service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:35.921887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcafeb-7e80-4c1e-b1a7-0eb583f134ac service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:35.922164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcafeb-7e80-4c1e-b1a7-0eb583f134ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:35.932218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcafeb-7e80-4c1e-b1a7-0eb583f134ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:35.933689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcafeb-7e80-4c1e-b1a7-0eb583f134ac service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:35.951384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 979/1924] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:36.466726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59b8d5-ebd5-4486-8e94-568b1b666cd7 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:36.468158 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6c59b8d5-ebd5-4486-8e94-568b1b666cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:36.488044 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c59b8d5-ebd5-4486-8e94-568b1b666cd7 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:36.500400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:36.501045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:36.501712 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:36.502226 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] PATH_INFO: `/v3/projects/81b2e4b2b5a94284a76fa794e402b583` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:36.503556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] RBAC: Authorizing `identity:delete_project(project_id=81b2e4b2b5a94284a76fa794e402b583)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:36.505596 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:36.710291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81b2e4b2b5a94284a76fa794e402b583', 'request_id': 'req-1c7927c5-60f5-42b3-8aef-49e768bd923f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:36.717196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:36.718777 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81b2e4b2b5a94284a76fa794e402b583', 'request_id': 'req-1c7927c5-60f5-42b3-8aef-49e768bd923f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:36.730900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81b2e4b2b5a94284a76fa794e402b583', 'request_id': 'req-1c7927c5-60f5-42b3-8aef-49e768bd923f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:36.823734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] The token cache is being invalidate because project 81b2e4b2b5a94284a76fa794e402b583 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:36.824300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:36.828422 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:36.830644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 946/1925] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:36 2022] DELETE /identity/v3/projects/81b2e4b2b5a94284a76fa794e402b583 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:36.835946 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcafeb-7e80-4c1e-b1a7-0eb583f134ac service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:36.849870 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcafeb-7e80-4c1e-b1a7-0eb583f134ac service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:36.954117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7ccaf-d56d-4b75-abf5-81c813132d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:36.955685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7ccaf-d56d-4b75-abf5-81c813132d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:36.956116 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7ccaf-d56d-4b75-abf5-81c813132d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:36.956554 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7ccaf-d56d-4b75-abf5-81c813132d05 admin admin] PATH_INFO: `/v3/users/e07851467d274f5c8f8e3cec27719c49` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:36.958172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7ccaf-d56d-4b75-abf5-81c813132d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e07851467d274f5c8f8e3cec27719c49)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:36.960344 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7ccaf-d56d-4b75-abf5-81c813132d05 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:37.088102 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:37.181724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7f7ccaf-d56d-4b75-abf5-81c813132d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e07851467d274f5c8f8e3cec27719c49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:37.261017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7f7ccaf-d56d-4b75-abf5-81c813132d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e07851467d274f5c8f8e3cec27719c49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:37.277154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7f7ccaf-d56d-4b75-abf5-81c813132d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e07851467d274f5c8f8e3cec27719c49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:37.277905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7f7ccaf-d56d-4b75-abf5-81c813132d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e07851467d274f5c8f8e3cec27719c49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:37.278521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7f7ccaf-d56d-4b75-abf5-81c813132d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e07851467d274f5c8f8e3cec27719c49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:37.319208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 980/1926] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:36 2022] DELETE /identity/v3/users/e07851467d274f5c8f8e3cec27719c49 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:37.939654 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:37.942975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:37.976058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7927c5-60f5-42b3-8aef-49e768bd923f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:38.077112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:38.078539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:38.078832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:38.079103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23 admin admin] PATH_INFO: `/v3/projects/ec55bec566904ad083decc70ee7003e5` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:38.080300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec55bec566904ad083decc70ee7003e5)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:38.082250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:38.318629 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec55bec566904ad083decc70ee7003e5', 'request_id': 'req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:38.331537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec55bec566904ad083decc70ee7003e5', 'request_id': 'req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:38.341012 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec55bec566904ad083decc70ee7003e5', 'request_id': 'req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:38.457150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23 admin admin] The token cache is being invalidate because project ec55bec566904ad083decc70ee7003e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:38.457532 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:38.461316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-fe467534-7b48-4ecb-a2b8-4e63cd3b4a23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:38.463703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 947/1927] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:37 2022] DELETE /identity/v3/projects/ec55bec566904ad083decc70ee7003e5 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:38.468526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c96fc-acd8-49b2-bed1-1833e90b5690 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:38.469672 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c96fc-acd8-49b2-bed1-1833e90b5690 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:38.470058 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c96fc-acd8-49b2-bed1-1833e90b5690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:38.496199 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-9e9c96fc-acd8-49b2-bed1-1833e90b5690 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:38.497413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9e9c96fc-acd8-49b2-bed1-1833e90b5690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:38.622789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 981/1928] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:38.622947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91637e-146d-40e4-b4c4-75c024597767 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:38.622947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91637e-146d-40e4-b4c4-75c024597767 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:38.622947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91637e-146d-40e4-b4c4-75c024597767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:38.663256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-9a91637e-146d-40e4-b4c4-75c024597767 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:38.665392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-9a91637e-146d-40e4-b4c4-75c024597767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:38.682660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c96fc-acd8-49b2-bed1-1833e90b5690 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:38.687528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 948/1929] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:38.697592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c96fc-acd8-49b2-bed1-1833e90b5690 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:38.703368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9e0d4-4d40-46f2-98a1-1ef12791185c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:38.707859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9e0d4-4d40-46f2-98a1-1ef12791185c admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:38.710613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9e0d4-4d40-46f2-98a1-1ef12791185c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:38.710901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9e0d4-4d40-46f2-98a1-1ef12791185c admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:38.711607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9e0d4-4d40-46f2-98a1-1ef12791185c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:38.711895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9e0d4-4d40-46f2-98a1-1ef12791185c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:38.713513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9e0d4-4d40-46f2-98a1-1ef12791185c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:38.755808 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 982/1930] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:56:38.759889 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91637e-146d-40e4-b4c4-75c024597767 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:38.770976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91637e-146d-40e4-b4c4-75c024597767 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:38.778572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:38.788820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83685efe-d12c-4659-90cc-3b8ba5204674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:38.789794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83685efe-d12c-4659-90cc-3b8ba5204674 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:38.790284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83685efe-d12c-4659-90cc-3b8ba5204674 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:38.790737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83685efe-d12c-4659-90cc-3b8ba5204674 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:38.791644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83685efe-d12c-4659-90cc-3b8ba5204674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:38.794727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83685efe-d12c-4659-90cc-3b8ba5204674 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:38.922891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9e0d4-4d40-46f2-98a1-1ef12791185c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:38.924087 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 949/1931] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:38 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:56:38.940955 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9e0d4-4d40-46f2-98a1-1ef12791185c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:39.045289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e0495-391d-412e-a0bc-c5c5fbd1c02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:39.045973 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e0495-391d-412e-a0bc-c5c5fbd1c02f admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:39.046603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e0495-391d-412e-a0bc-c5c5fbd1c02f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:39.046603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e0495-391d-412e-a0bc-c5c5fbd1c02f admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:39.051026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e0495-391d-412e-a0bc-c5c5fbd1c02f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:39.051026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729e0495-391d-412e-a0bc-c5c5fbd1c02f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:39.163199 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 983/1932] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:38 2022] POST /identity/v3/users => generated 283 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:39.167082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83685efe-d12c-4659-90cc-3b8ba5204674 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:39.183751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83685efe-d12c-4659-90cc-3b8ba5204674 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:39.189116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d444-45fa-48fa-aa64-0f4bee4fc91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:39.190063 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d444-45fa-48fa-aa64-0f4bee4fc91b admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:39.194491 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d444-45fa-48fa-aa64-0f4bee4fc91b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:39.194491 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201d444-45fa-48fa-aa64-0f4bee4fc91b admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:39.194491 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201d444-45fa-48fa-aa64-0f4bee4fc91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:39.194491 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201d444-45fa-48fa-aa64-0f4bee4fc91b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:39.205793 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 950/1933] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:56:39.210064 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e0495-391d-412e-a0bc-c5c5fbd1c02f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:39.221711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e0495-391d-412e-a0bc-c5c5fbd1c02f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:39.226792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276e041-22ef-452a-87d7-91efcebfb60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:39.227558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276e041-22ef-452a-87d7-91efcebfb60e admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:39.227909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276e041-22ef-452a-87d7-91efcebfb60e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:39.228190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276e041-22ef-452a-87d7-91efcebfb60e admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:39.228902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276e041-22ef-452a-87d7-91efcebfb60e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:39.230807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276e041-22ef-452a-87d7-91efcebfb60e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:39.243056 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 984/1934] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:39.247379 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d444-45fa-48fa-aa64-0f4bee4fc91b admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:39.258416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201d444-45fa-48fa-aa64-0f4bee4fc91b admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:39.266271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60beaee9-3cb3-4b18-9157-70c6a0b43f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:39.266949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60beaee9-3cb3-4b18-9157-70c6a0b43f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:39.267762 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60beaee9-3cb3-4b18-9157-70c6a0b43f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:39.268044 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60beaee9-3cb3-4b18-9157-70c6a0b43f9a admin admin] PATH_INFO: `/v3/projects/1aefb4089113410da2d3cc684bc20b02/users/01dbefa7fffe4eba9f1b0eb19390735f/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:39.285048 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60beaee9-3cb3-4b18-9157-70c6a0b43f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aefb4089113410da2d3cc684bc20b02, user_id=01dbefa7fffe4eba9f1b0eb19390735f, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:39.287681 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60beaee9-3cb3-4b18-9157-70c6a0b43f9a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:39.420696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 951/1935] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:39 2022] PUT /identity/v3/projects/1aefb4089113410da2d3cc684bc20b02/users/01dbefa7fffe4eba9f1b0eb19390735f/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:39.431665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72305ee8-a52d-400f-97a9-266ab519209e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:39.432346 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72305ee8-a52d-400f-97a9-266ab519209e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:39.432346 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72305ee8-a52d-400f-97a9-266ab519209e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:39.471953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:39.483645 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-72305ee8-a52d-400f-97a9-266ab519209e None None] MFA Rules not processed for user `01dbefa7fffe4eba9f1b0eb19390735f`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:39.484940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-72305ee8-a52d-400f-97a9-266ab519209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:39.573854 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 985/1936] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:39.577490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60beaee9-3cb3-4b18-9157-70c6a0b43f9a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:39.665090 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60beaee9-3cb3-4b18-9157-70c6a0b43f9a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:39.674860 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2713a5a-57b3-473a-8c75-d2ec49752c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:39.675609 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2713a5a-57b3-473a-8c75-d2ec49752c22 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:39.675953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2713a5a-57b3-473a-8c75-d2ec49752c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:39.676298 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2713a5a-57b3-473a-8c75-d2ec49752c22 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:39.677011 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2713a5a-57b3-473a-8c75-d2ec49752c22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:39.679223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2713a5a-57b3-473a-8c75-d2ec49752c22 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:39.830359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 952/1937] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:39 2022] POST /identity/v3/projects => generated 365 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:39.834446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72305ee8-a52d-400f-97a9-266ab519209e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:39.845356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72305ee8-a52d-400f-97a9-266ab519209e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:39.946382 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086c5d5-00bb-409e-92f2-2c358e38feaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:39.947498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086c5d5-00bb-409e-92f2-2c358e38feaa admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:39.947498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086c5d5-00bb-409e-92f2-2c358e38feaa admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:39.947764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086c5d5-00bb-409e-92f2-2c358e38feaa admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:39.948370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086c5d5-00bb-409e-92f2-2c358e38feaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:39.951282 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086c5d5-00bb-409e-92f2-2c358e38feaa admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:40.129249 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 986/1938] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:39 2022] POST /identity/v3/users => generated 289 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:40.133439 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2713a5a-57b3-473a-8c75-d2ec49752c22 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:40.145720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2713a5a-57b3-473a-8c75-d2ec49752c22 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:40.151423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa4415-efbf-44a2-86a9-d44b9cb24e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:40.152247 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa4415-efbf-44a2-86a9-d44b9cb24e9e admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:40.152530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa4415-efbf-44a2-86a9-d44b9cb24e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:40.152819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aa4415-efbf-44a2-86a9-d44b9cb24e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:40.153443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa4415-efbf-44a2-86a9-d44b9cb24e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:40.161551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aa4415-efbf-44a2-86a9-d44b9cb24e9e admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:40.174418 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 953/1939] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:56:40.178559 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086c5d5-00bb-409e-92f2-2c358e38feaa admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:40.194786 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086c5d5-00bb-409e-92f2-2c358e38feaa admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:40.200432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544bca5-3242-4187-8366-56310c3807f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:40.201219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544bca5-3242-4187-8366-56310c3807f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:40.201504 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544bca5-3242-4187-8366-56310c3807f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:40.201779 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544bca5-3242-4187-8366-56310c3807f1 admin admin] PATH_INFO: `/v3/projects/189d3d2f10bc42f5864055feec04cb4c/users/018194ce21184e24a557c1b9f3de6e5b/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:40.226570 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544bca5-3242-4187-8366-56310c3807f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=189d3d2f10bc42f5864055feec04cb4c, user_id=018194ce21184e24a557c1b9f3de6e5b, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:40.229972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544bca5-3242-4187-8366-56310c3807f1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:40.348898 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 987/1940] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:40 2022] PUT /identity/v3/projects/189d3d2f10bc42f5864055feec04cb4c/users/018194ce21184e24a557c1b9f3de6e5b/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:40.355845 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa4415-efbf-44a2-86a9-d44b9cb24e9e admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:40.371886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aa4415-efbf-44a2-86a9-d44b9cb24e9e admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:40.490589 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096234d7-37de-4303-85f4-e0ae3a9386fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:40.491559 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096234d7-37de-4303-85f4-e0ae3a9386fe admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:40.492242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096234d7-37de-4303-85f4-e0ae3a9386fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:40.492910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096234d7-37de-4303-85f4-e0ae3a9386fe admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:40.495804 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096234d7-37de-4303-85f4-e0ae3a9386fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:40.501111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096234d7-37de-4303-85f4-e0ae3a9386fe admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:40.517265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 954/1941] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:40.521364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544bca5-3242-4187-8366-56310c3807f1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:40.536227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544bca5-3242-4187-8366-56310c3807f1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:40.547971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4de66-3f8c-4323-aded-2e336e1b7ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:40.549312 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4de66-3f8c-4323-aded-2e336e1b7ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:40.549664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4de66-3f8c-4323-aded-2e336e1b7ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:40.550467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4de66-3f8c-4323-aded-2e336e1b7ecd admin admin] PATH_INFO: `/v3/projects/189d3d2f10bc42f5864055feec04cb4c/users/018194ce21184e24a557c1b9f3de6e5b/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:40.554876 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4de66-3f8c-4323-aded-2e336e1b7ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=189d3d2f10bc42f5864055feec04cb4c, user_id=018194ce21184e24a557c1b9f3de6e5b, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:40.561982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4de66-3f8c-4323-aded-2e336e1b7ecd admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:40.597369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 988/1942] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:40 2022] PUT /identity/v3/projects/189d3d2f10bc42f5864055feec04cb4c/users/018194ce21184e24a557c1b9f3de6e5b/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:40.610787 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fc1b8-d5f5-43a6-9c74-372aa91c35c0 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:40.611124 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fc1b8-d5f5-43a6-9c74-372aa91c35c0 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:40.611459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555fc1b8-d5f5-43a6-9c74-372aa91c35c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:40.660490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-555fc1b8-d5f5-43a6-9c74-372aa91c35c0 None None] MFA Rules not processed for user `018194ce21184e24a557c1b9f3de6e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:40.662941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-555fc1b8-d5f5-43a6-9c74-372aa91c35c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:40.809950 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 955/1943] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:41.022989 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4de66-3f8c-4323-aded-2e336e1b7ecd admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:41.025406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fdb4de66-3f8c-4323-aded-2e336e1b7ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:41.094792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4de66-3f8c-4323-aded-2e336e1b7ecd admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:41.098396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c41759-811f-4e44-916a-c06985edc37c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:41.099127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c41759-811f-4e44-916a-c06985edc37c service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:41.099500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c41759-811f-4e44-916a-c06985edc37c service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:41.099841 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c41759-811f-4e44-916a-c06985edc37c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:41.110428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c41759-811f-4e44-916a-c06985edc37c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:41.112264 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c41759-811f-4e44-916a-c06985edc37c service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:41.126316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 989/1944] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:41.534335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fc1b8-d5f5-43a6-9c74-372aa91c35c0 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:41.536481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-555fc1b8-d5f5-43a6-9c74-372aa91c35c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:41.539768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:41.551912 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555fc1b8-d5f5-43a6-9c74-372aa91c35c0 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:41.555655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df5fe2-2397-4c48-b0d2-f1f8676a3d45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:41.556270 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df5fe2-2397-4c48-b0d2-f1f8676a3d45 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:41.556558 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df5fe2-2397-4c48-b0d2-f1f8676a3d45 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:41.556818 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df5fe2-2397-4c48-b0d2-f1f8676a3d45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:41.632294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df5fe2-2397-4c48-b0d2-f1f8676a3d45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:41.634522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df5fe2-2397-4c48-b0d2-f1f8676a3d45 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:41.651359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 956/1945] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:41.769185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:41.977621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d670538-dddb-4d6d-9366-5bf3560ec970 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:41.978213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d670538-dddb-4d6d-9366-5bf3560ec970 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:41.978489 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d670538-dddb-4d6d-9366-5bf3560ec970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:42.016145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6d670538-dddb-4d6d-9366-5bf3560ec970 None None] MFA Rules not processed for user `0ebafc08399c4853959800883742eb78`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:42.019549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6d670538-dddb-4d6d-9366-5bf3560ec970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:42.151883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 990/1946] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:42.154153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df5fe2-2397-4c48-b0d2-f1f8676a3d45 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:42.155778 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-08df5fe2-2397-4c48-b0d2-f1f8676a3d45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:42.355410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df5fe2-2397-4c48-b0d2-f1f8676a3d45 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:42.379353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0184fc2-1d9c-411a-b9fc-948e1f7683e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:42.380068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0184fc2-1d9c-411a-b9fc-948e1f7683e4 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:42.382326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0184fc2-1d9c-411a-b9fc-948e1f7683e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:42.382624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0184fc2-1d9c-411a-b9fc-948e1f7683e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:42.383453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0184fc2-1d9c-411a-b9fc-948e1f7683e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:42.396911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0184fc2-1d9c-411a-b9fc-948e1f7683e4 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:42.574611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 957/1947] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:42 2022] POST /identity/v3/projects => generated 383 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:42.577538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d670538-dddb-4d6d-9366-5bf3560ec970 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:42.598139 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d670538-dddb-4d6d-9366-5bf3560ec970 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:42.733208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75becd8-6928-4709-9fca-aa8c4ecf0273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:42.733208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75becd8-6928-4709-9fca-aa8c4ecf0273 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:42.733208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75becd8-6928-4709-9fca-aa8c4ecf0273 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:42.733645 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75becd8-6928-4709-9fca-aa8c4ecf0273 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:42.734167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75becd8-6928-4709-9fca-aa8c4ecf0273 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:42.736681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75becd8-6928-4709-9fca-aa8c4ecf0273 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:42.843347 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 991/1948] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:42 2022] POST /identity/v3/users => generated 298 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:42.848560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0184fc2-1d9c-411a-b9fc-948e1f7683e4 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:42.889838 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0184fc2-1d9c-411a-b9fc-948e1f7683e4 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:42.896578 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac08718-37c4-472d-bf9f-13e8284a1718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:42.897501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac08718-37c4-472d-bf9f-13e8284a1718 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:42.897972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac08718-37c4-472d-bf9f-13e8284a1718 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:42.898244 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac08718-37c4-472d-bf9f-13e8284a1718 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:42.898893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac08718-37c4-472d-bf9f-13e8284a1718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:42.900503 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac08718-37c4-472d-bf9f-13e8284a1718 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:42.922826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 958/1949] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:42 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:42.928801 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75becd8-6928-4709-9fca-aa8c4ecf0273 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:42.947242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75becd8-6928-4709-9fca-aa8c4ecf0273 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:42.958977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98928aa-5454-4f0f-bb52-9ced5dffb91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:42.961928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98928aa-5454-4f0f-bb52-9ced5dffb91c admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:42.961928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98928aa-5454-4f0f-bb52-9ced5dffb91c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:42.961928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98928aa-5454-4f0f-bb52-9ced5dffb91c admin admin] PATH_INFO: `/v3/projects/64fbaaef29364174a40d699e0b9458af/users/5ce3c93ac7d04f6ea76fd82a2bc810cb/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:43.026096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98928aa-5454-4f0f-bb52-9ced5dffb91c admin admin] RBAC: Authorizing `identity:create_grant(project_id=64fbaaef29364174a40d699e0b9458af, user_id=5ce3c93ac7d04f6ea76fd82a2bc810cb, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:43.026096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98928aa-5454-4f0f-bb52-9ced5dffb91c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:43.175924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 992/1950] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:42 2022] PUT /identity/v3/projects/64fbaaef29364174a40d699e0b9458af/users/5ce3c93ac7d04f6ea76fd82a2bc810cb/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:43.180033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac08718-37c4-472d-bf9f-13e8284a1718 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:43.191793 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac08718-37c4-472d-bf9f-13e8284a1718 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:43.312134 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e407df-6799-4aa5-b95a-00a050bdba2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:43.312761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e407df-6799-4aa5-b95a-00a050bdba2a admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:43.313037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e407df-6799-4aa5-b95a-00a050bdba2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:43.313306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e407df-6799-4aa5-b95a-00a050bdba2a admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:43.313978 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e407df-6799-4aa5-b95a-00a050bdba2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:43.316491 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e407df-6799-4aa5-b95a-00a050bdba2a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:43.342249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 959/1951] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:43.346668 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98928aa-5454-4f0f-bb52-9ced5dffb91c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:43.372448 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98928aa-5454-4f0f-bb52-9ced5dffb91c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:43.381370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f8aff-f427-446d-996d-69890730a0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:43.382408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f8aff-f427-446d-996d-69890730a0a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:43.383010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f8aff-f427-446d-996d-69890730a0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:43.384960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f8aff-f427-446d-996d-69890730a0a9 admin admin] PATH_INFO: `/v3/projects/64fbaaef29364174a40d699e0b9458af/users/5ce3c93ac7d04f6ea76fd82a2bc810cb/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:43.389187 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f8aff-f427-446d-996d-69890730a0a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64fbaaef29364174a40d699e0b9458af, user_id=5ce3c93ac7d04f6ea76fd82a2bc810cb, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:43.406559 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f8aff-f427-446d-996d-69890730a0a9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:43.408504 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 993/1952] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:43 2022] PUT /identity/v3/projects/64fbaaef29364174a40d699e0b9458af/users/5ce3c93ac7d04f6ea76fd82a2bc810cb/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:43.549900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e407df-6799-4aa5-b95a-00a050bdba2a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:43.551738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b3e407df-6799-4aa5-b95a-00a050bdba2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:43.569734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e407df-6799-4aa5-b95a-00a050bdba2a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:43.651660 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942caada-5f83-4185-ac78-c88b77abfc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:43.652516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942caada-5f83-4185-ac78-c88b77abfc33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:43.653086 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942caada-5f83-4185-ac78-c88b77abfc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:43.653474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942caada-5f83-4185-ac78-c88b77abfc33 admin admin] PATH_INFO: `/v3/users/07c601b050c4428381ccf74d805f7bf6` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:43.655242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942caada-5f83-4185-ac78-c88b77abfc33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07c601b050c4428381ccf74d805f7bf6)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:43.658861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942caada-5f83-4185-ac78-c88b77abfc33 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:43.977216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-942caada-5f83-4185-ac78-c88b77abfc33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07c601b050c4428381ccf74d805f7bf6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:43.990271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-942caada-5f83-4185-ac78-c88b77abfc33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07c601b050c4428381ccf74d805f7bf6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:44.103890 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-942caada-5f83-4185-ac78-c88b77abfc33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07c601b050c4428381ccf74d805f7bf6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:44.104028 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:44.107009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-942caada-5f83-4185-ac78-c88b77abfc33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07c601b050c4428381ccf74d805f7bf6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:44.116142 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-942caada-5f83-4185-ac78-c88b77abfc33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07c601b050c4428381ccf74d805f7bf6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:44.122588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 960/1953] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:43 2022] DELETE /identity/v3/users/07c601b050c4428381ccf74d805f7bf6 => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:44.127134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5ca4a-c7fc-432d-9734-050886ca8639 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:44.127445 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5ca4a-c7fc-432d-9734-050886ca8639 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:44.127721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5ca4a-c7fc-432d-9734-050886ca8639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:44.149001 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4eb5ca4a-c7fc-432d-9734-050886ca8639 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:44.150252 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4eb5ca4a-c7fc-432d-9734-050886ca8639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:44.263843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 994/1954] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:44.278851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942caada-5f83-4185-ac78-c88b77abfc33 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:44.282164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-942caada-5f83-4185-ac78-c88b77abfc33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:44.382692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942caada-5f83-4185-ac78-c88b77abfc33 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:44.388441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0fc23-e7db-48e3-b9db-ae617517d813 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:44.389928 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0fc23-e7db-48e3-b9db-ae617517d813 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:44.390274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0fc23-e7db-48e3-b9db-ae617517d813 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:44.394066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0fc23-e7db-48e3-b9db-ae617517d813 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:44.406847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0fc23-e7db-48e3-b9db-ae617517d813 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:44.408413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0fc23-e7db-48e3-b9db-ae617517d813 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:44.434510 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 961/1955] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:45.039811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5ca4a-c7fc-432d-9734-050886ca8639 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:45.041329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4eb5ca4a-c7fc-432d-9734-050886ca8639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:45.060281 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5ca4a-c7fc-432d-9734-050886ca8639 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:45.066630 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:45.067462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:45.067746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:45.068021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] PATH_INFO: `/v3/projects/247adb0616cb456bb31122a8921cdf73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:45.070219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] RBAC: Authorizing `identity:delete_project(project_id=247adb0616cb456bb31122a8921cdf73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:45.072179 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:45.253460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '247adb0616cb456bb31122a8921cdf73', 'request_id': 'req-ca7d006a-1fc4-4db5-bd46-98fccb45695e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:45.305916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:45.352103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '247adb0616cb456bb31122a8921cdf73', 'request_id': 'req-ca7d006a-1fc4-4db5-bd46-98fccb45695e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:45.359649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '247adb0616cb456bb31122a8921cdf73', 'request_id': 'req-ca7d006a-1fc4-4db5-bd46-98fccb45695e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:45.376235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] The token cache is being invalidate because project 247adb0616cb456bb31122a8921cdf73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:45.377235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:45.378229 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:45.384616 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0fc23-e7db-48e3-b9db-ae617517d813 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:45.393634 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 995/1956] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:45 2022] DELETE /identity/v3/projects/247adb0616cb456bb31122a8921cdf73 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:45.400248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0fc23-e7db-48e3-b9db-ae617517d813 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:45.476857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f0b42c-53f7-4b4e-97c2-8604c60f5aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:45.477863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0b42c-53f7-4b4e-97c2-8604c60f5aae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:45.479231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0b42c-53f7-4b4e-97c2-8604c60f5aae admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:45.479678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f0b42c-53f7-4b4e-97c2-8604c60f5aae admin admin] PATH_INFO: `/v3/users/fc95d77e50fb48e09224a91e47f29feb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:45.481297 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0b42c-53f7-4b4e-97c2-8604c60f5aae admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc95d77e50fb48e09224a91e47f29feb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:45.484414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f0b42c-53f7-4b4e-97c2-8604c60f5aae admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:45.739333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e4f0b42c-53f7-4b4e-97c2-8604c60f5aae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc95d77e50fb48e09224a91e47f29feb'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:45.753857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e4f0b42c-53f7-4b4e-97c2-8604c60f5aae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc95d77e50fb48e09224a91e47f29feb'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:45.865971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e4f0b42c-53f7-4b4e-97c2-8604c60f5aae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc95d77e50fb48e09224a91e47f29feb'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:45.866848 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e4f0b42c-53f7-4b4e-97c2-8604c60f5aae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc95d77e50fb48e09224a91e47f29feb'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:45.899227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e4f0b42c-53f7-4b4e-97c2-8604c60f5aae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc95d77e50fb48e09224a91e47f29feb'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:45.911583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 962/1957] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:45 2022] DELETE /identity/v3/users/fc95d77e50fb48e09224a91e47f29feb => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:46.063538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:46.546588 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:46.553438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:46.574610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7d006a-1fc4-4db5-bd46-98fccb45695e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:46.654672 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bfa93b-1358-42e8-9f62-8682c5f72705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:46.655473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfa93b-1358-42e8-9f62-8682c5f72705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:46.656014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfa93b-1358-42e8-9f62-8682c5f72705 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:46.656305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfa93b-1358-42e8-9f62-8682c5f72705 admin admin] PATH_INFO: `/v3/projects/034ba9252ef549ad84b9a4f24a7f115b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:46.657573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfa93b-1358-42e8-9f62-8682c5f72705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=034ba9252ef549ad84b9a4f24a7f115b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:46.659805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bfa93b-1358-42e8-9f62-8682c5f72705 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:46.852691 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-14bfa93b-1358-42e8-9f62-8682c5f72705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '034ba9252ef549ad84b9a4f24a7f115b', 'request_id': 'req-14bfa93b-1358-42e8-9f62-8682c5f72705'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:46.991326 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-14bfa93b-1358-42e8-9f62-8682c5f72705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '034ba9252ef549ad84b9a4f24a7f115b', 'request_id': 'req-14bfa93b-1358-42e8-9f62-8682c5f72705'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:46.999065 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-14bfa93b-1358-42e8-9f62-8682c5f72705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '034ba9252ef549ad84b9a4f24a7f115b', 'request_id': 'req-14bfa93b-1358-42e8-9f62-8682c5f72705'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:47.018468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-14bfa93b-1358-42e8-9f62-8682c5f72705 admin admin] The token cache is being invalidate because project 034ba9252ef549ad84b9a4f24a7f115b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:47.018978 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-14bfa93b-1358-42e8-9f62-8682c5f72705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:47.019776 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-14bfa93b-1358-42e8-9f62-8682c5f72705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:47.023228 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 996/1958] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:46 2022] DELETE /identity/v3/projects/034ba9252ef549ad84b9a4f24a7f115b => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:47.031051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16a327-cefa-4b75-94cf-9a48e37a1f18 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:47.031505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16a327-cefa-4b75-94cf-9a48e37a1f18 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:47.031803 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16a327-cefa-4b75-94cf-9a48e37a1f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:47.063493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-0d16a327-cefa-4b75-94cf-9a48e37a1f18 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:47.064784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0d16a327-cefa-4b75-94cf-9a48e37a1f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:47.132583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 963/1959] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:47.140304 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bc6ca-04a9-4beb-b62f-efe83c488f1f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:47.140795 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bc6ca-04a9-4beb-b62f-efe83c488f1f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:47.141218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9bc6ca-04a9-4beb-b62f-efe83c488f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:47.199304 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-ee9bc6ca-04a9-4beb-b62f-efe83c488f1f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:47.200922 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ee9bc6ca-04a9-4beb-b62f-efe83c488f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:47.208878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 997/1960] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:47.213068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16a327-cefa-4b75-94cf-9a48e37a1f18 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:47.223647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16a327-cefa-4b75-94cf-9a48e37a1f18 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:47.229155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9fd28-9aec-4fb2-8eec-23e5d9e17268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:47.230243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9fd28-9aec-4fb2-8eec-23e5d9e17268 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:47.230690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9fd28-9aec-4fb2-8eec-23e5d9e17268 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:47.235394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:47.235394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9fd28-9aec-4fb2-8eec-23e5d9e17268 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:47.235394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9fd28-9aec-4fb2-8eec-23e5d9e17268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:47.235394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9fd28-9aec-4fb2-8eec-23e5d9e17268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:47.235394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9fd28-9aec-4fb2-8eec-23e5d9e17268 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:47.250042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 964/1961] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:56:47.254858 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bc6ca-04a9-4beb-b62f-efe83c488f1f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:47.264493 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9bc6ca-04a9-4beb-b62f-efe83c488f1f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:47.273658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67912f1-cf63-4c07-8ba2-1584a85bda6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:47.274451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67912f1-cf63-4c07-8ba2-1584a85bda6d admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:47.274731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67912f1-cf63-4c07-8ba2-1584a85bda6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:47.274999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67912f1-cf63-4c07-8ba2-1584a85bda6d admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:47.275825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67912f1-cf63-4c07-8ba2-1584a85bda6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:47.277736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67912f1-cf63-4c07-8ba2-1584a85bda6d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:47.416076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 998/1962] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:47 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:47.420126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9fd28-9aec-4fb2-8eec-23e5d9e17268 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:47.432243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9fd28-9aec-4fb2-8eec-23e5d9e17268 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:47.531562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c17c6-6d17-4a01-a6ae-0e291a5c7a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:47.531562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c17c6-6d17-4a01-a6ae-0e291a5c7a21 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:47.531562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c17c6-6d17-4a01-a6ae-0e291a5c7a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:47.531562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10c17c6-6d17-4a01-a6ae-0e291a5c7a21 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:47.531912 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c17c6-6d17-4a01-a6ae-0e291a5c7a21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:47.531912 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10c17c6-6d17-4a01-a6ae-0e291a5c7a21 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:47.621852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 965/1963] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:47 2022] POST /identity/v3/users => generated 291 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:56:47.623862 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67912f1-cf63-4c07-8ba2-1584a85bda6d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:47.638419 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67912f1-cf63-4c07-8ba2-1584a85bda6d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:47.645987 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb207a-b410-4d9f-844f-920f7570f5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:47.647237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb207a-b410-4d9f-844f-920f7570f5ec admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:47.647804 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb207a-b410-4d9f-844f-920f7570f5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:47.648248 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb207a-b410-4d9f-844f-920f7570f5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:47.650768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb207a-b410-4d9f-844f-920f7570f5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:47.652908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb207a-b410-4d9f-844f-920f7570f5ec admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:47.668079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 999/1964] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:47.672419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c17c6-6d17-4a01-a6ae-0e291a5c7a21 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:47.690876 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10c17c6-6d17-4a01-a6ae-0e291a5c7a21 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:47.699033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aea09c-347f-4d91-97da-b39868ba2d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:47.699675 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aea09c-347f-4d91-97da-b39868ba2d58 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:47.699936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aea09c-347f-4d91-97da-b39868ba2d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:47.700204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aea09c-347f-4d91-97da-b39868ba2d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:47.700852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aea09c-347f-4d91-97da-b39868ba2d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:47.704252 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aea09c-347f-4d91-97da-b39868ba2d58 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:47.722424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 966/1965] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:47.727758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb207a-b410-4d9f-844f-920f7570f5ec admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:47.759912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb207a-b410-4d9f-844f-920f7570f5ec admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:47.763166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:47.782831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fda011-9895-4897-96b2-5f182e58cc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:47.784358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fda011-9895-4897-96b2-5f182e58cc35 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:47.784836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fda011-9895-4897-96b2-5f182e58cc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:47.785283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fda011-9895-4897-96b2-5f182e58cc35 admin admin] PATH_INFO: `/v3/projects/8b20d487ab6246bbb42631a5dcf5d16c/users/7ed1e55827d041bf872dd067e1bea402/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:47.816905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fda011-9895-4897-96b2-5f182e58cc35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b20d487ab6246bbb42631a5dcf5d16c, user_id=7ed1e55827d041bf872dd067e1bea402, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:47.820657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fda011-9895-4897-96b2-5f182e58cc35 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:47.926753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1000/1966] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:47 2022] PUT /identity/v3/projects/8b20d487ab6246bbb42631a5dcf5d16c/users/7ed1e55827d041bf872dd067e1bea402/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:47.943592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3cddc-2848-4111-8378-05f2d2d39dbb None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:47.943945 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3cddc-2848-4111-8378-05f2d2d39dbb None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:47.944151 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd3cddc-2848-4111-8378-05f2d2d39dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:47.983352 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-bdd3cddc-2848-4111-8378-05f2d2d39dbb None None] MFA Rules not processed for user `7ed1e55827d041bf872dd067e1bea402`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:47.984991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bdd3cddc-2848-4111-8378-05f2d2d39dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:48.068512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 967/1967] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:48.073440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fda011-9895-4897-96b2-5f182e58cc35 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:48.141449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fda011-9895-4897-96b2-5f182e58cc35 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:48.148145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f247f07-84e0-4418-bc4c-07a4fdfd18a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:48.148494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f247f07-84e0-4418-bc4c-07a4fdfd18a0 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:48.148771 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f247f07-84e0-4418-bc4c-07a4fdfd18a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:48.149039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f247f07-84e0-4418-bc4c-07a4fdfd18a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:48.149753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f247f07-84e0-4418-bc4c-07a4fdfd18a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:48.151672 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f247f07-84e0-4418-bc4c-07a4fdfd18a0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:48.293430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3cddc-2848-4111-8378-05f2d2d39dbb None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:48.302551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1001/1968] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:48 2022] POST /identity/v3/projects => generated 381 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:56:48.309579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd3cddc-2848-4111-8378-05f2d2d39dbb None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:48.411127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefd818-4e0c-4917-87d3-68ab56aaef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:48.411742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefd818-4e0c-4917-87d3-68ab56aaef65 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:48.412019 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefd818-4e0c-4917-87d3-68ab56aaef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:48.412650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefd818-4e0c-4917-87d3-68ab56aaef65 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:48.413336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefd818-4e0c-4917-87d3-68ab56aaef65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:48.415308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cefd818-4e0c-4917-87d3-68ab56aaef65 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:48.554171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 968/1969] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:48 2022] POST /identity/v3/users => generated 297 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:48.558975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f247f07-84e0-4418-bc4c-07a4fdfd18a0 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:48.579948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f247f07-84e0-4418-bc4c-07a4fdfd18a0 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:48.588039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c1b4e-43f2-4225-a118-8e894669909f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:48.589627 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c1b4e-43f2-4225-a118-8e894669909f admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:48.590100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c1b4e-43f2-4225-a118-8e894669909f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:48.591083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684c1b4e-43f2-4225-a118-8e894669909f admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:48.592686 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c1b4e-43f2-4225-a118-8e894669909f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:48.595051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684c1b4e-43f2-4225-a118-8e894669909f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:48.614635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1002/1970] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:48.619224 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefd818-4e0c-4917-87d3-68ab56aaef65 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:48.634886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefd818-4e0c-4917-87d3-68ab56aaef65 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:48.639837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfb2a9-1ca6-4d8d-91d6-52e492bfefda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:48.640588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb2a9-1ca6-4d8d-91d6-52e492bfefda admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:48.640936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb2a9-1ca6-4d8d-91d6-52e492bfefda admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:48.641282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb2a9-1ca6-4d8d-91d6-52e492bfefda admin admin] PATH_INFO: `/v3/projects/34cb4dd0716d4035919e84c750ad0ccb/users/6ccb9e0ad9a94525a6a4a2a1041349ba/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:48.657847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfb2a9-1ca6-4d8d-91d6-52e492bfefda admin admin] RBAC: Authorizing `identity:create_grant(project_id=34cb4dd0716d4035919e84c750ad0ccb, user_id=6ccb9e0ad9a94525a6a4a2a1041349ba, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:48.660972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfb2a9-1ca6-4d8d-91d6-52e492bfefda admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:48.744263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 969/1971] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:48 2022] PUT /identity/v3/projects/34cb4dd0716d4035919e84c750ad0ccb/users/6ccb9e0ad9a94525a6a4a2a1041349ba/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:48.748377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c1b4e-43f2-4225-a118-8e894669909f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:48.758714 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684c1b4e-43f2-4225-a118-8e894669909f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:48.841451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1f961-7fa4-4c41-801b-c05641a58ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:48.842461 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1f961-7fa4-4c41-801b-c05641a58ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:48.842741 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1f961-7fa4-4c41-801b-c05641a58ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:48.843013 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1f961-7fa4-4c41-801b-c05641a58ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:48.843744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1f961-7fa4-4c41-801b-c05641a58ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:48.845406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1f961-7fa4-4c41-801b-c05641a58ea7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:48.865259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1003/1972] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:48.875363 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfb2a9-1ca6-4d8d-91d6-52e492bfefda admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:48.889428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfb2a9-1ca6-4d8d-91d6-52e492bfefda admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:48.895499 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf35480-78fa-4643-a06d-37ba7b4ab4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:48.896266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf35480-78fa-4643-a06d-37ba7b4ab4be admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:48.896602 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf35480-78fa-4643-a06d-37ba7b4ab4be admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:48.897582 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf35480-78fa-4643-a06d-37ba7b4ab4be admin admin] PATH_INFO: `/v3/projects/34cb4dd0716d4035919e84c750ad0ccb/users/6ccb9e0ad9a94525a6a4a2a1041349ba/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:48.899109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf35480-78fa-4643-a06d-37ba7b4ab4be admin admin] RBAC: Authorizing `identity:create_grant(project_id=34cb4dd0716d4035919e84c750ad0ccb, user_id=6ccb9e0ad9a94525a6a4a2a1041349ba, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:48.902021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf35480-78fa-4643-a06d-37ba7b4ab4be admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:48.924459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 970/1973] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:48 2022] PUT /identity/v3/projects/34cb4dd0716d4035919e84c750ad0ccb/users/6ccb9e0ad9a94525a6a4a2a1041349ba/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:48.941879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6323834-46c7-4f89-95ab-a7c5c311bebf None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:48.945651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6323834-46c7-4f89-95ab-a7c5c311bebf None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:48.945651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6323834-46c7-4f89-95ab-a7c5c311bebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:48.962931 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:48.990229 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f6323834-46c7-4f89-95ab-a7c5c311bebf None None] MFA Rules not processed for user `6ccb9e0ad9a94525a6a4a2a1041349ba`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:48.991869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f6323834-46c7-4f89-95ab-a7c5c311bebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:49.095281 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1004/1974] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:49.107810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf35480-78fa-4643-a06d-37ba7b4ab4be admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:49.109864 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-fcf35480-78fa-4643-a06d-37ba7b4ab4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:49.211036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf35480-78fa-4643-a06d-37ba7b4ab4be admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:49.214770 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e440b-e1ac-4788-ad65-5995a7ae3698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:49.215377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e440b-e1ac-4788-ad65-5995a7ae3698 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:49.215652 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e440b-e1ac-4788-ad65-5995a7ae3698 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:49.215983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e440b-e1ac-4788-ad65-5995a7ae3698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:49.225412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e440b-e1ac-4788-ad65-5995a7ae3698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:49.226880 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e440b-e1ac-4788-ad65-5995a7ae3698 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:49.242135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 971/1975] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:49 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:49.484483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6323834-46c7-4f89-95ab-a7c5c311bebf None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:49.495949 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6323834-46c7-4f89-95ab-a7c5c311bebf None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:49.503655 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa945127-6991-44d5-b823-d11e8006bee4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:49.504301 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa945127-6991-44d5-b823-d11e8006bee4 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:49.504576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa945127-6991-44d5-b823-d11e8006bee4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:49.504875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa945127-6991-44d5-b823-d11e8006bee4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:49.570085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa945127-6991-44d5-b823-d11e8006bee4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:49.571659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa945127-6991-44d5-b823-d11e8006bee4 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:49.582689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1005/1976] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:49.702833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e440b-e1ac-4788-ad65-5995a7ae3698 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:49.704307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bd1e440b-e1ac-4788-ad65-5995a7ae3698 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:49.800565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e440b-e1ac-4788-ad65-5995a7ae3698 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:49.805842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4813d4-4bd6-4038-8b0f-540c442c2c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:49.806712 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4813d4-4bd6-4038-8b0f-540c442c2c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:49.807054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4813d4-4bd6-4038-8b0f-540c442c2c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:49.807399 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4813d4-4bd6-4038-8b0f-540c442c2c8b admin admin] PATH_INFO: `/v3/users/01dbefa7fffe4eba9f1b0eb19390735f` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:49.808637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4813d4-4bd6-4038-8b0f-540c442c2c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=01dbefa7fffe4eba9f1b0eb19390735f)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:49.810944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4813d4-4bd6-4038-8b0f-540c442c2c8b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:50.051183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1d4813d4-4bd6-4038-8b0f-540c442c2c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01dbefa7fffe4eba9f1b0eb19390735f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:50.064021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf395d4-952b-4d5d-b69c-511b9f87fc0c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:50.065062 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf395d4-952b-4d5d-b69c-511b9f87fc0c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:50.065144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf395d4-952b-4d5d-b69c-511b9f87fc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:50.072184 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1d4813d4-4bd6-4038-8b0f-540c442c2c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01dbefa7fffe4eba9f1b0eb19390735f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:50.116944 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-baf395d4-952b-4d5d-b69c-511b9f87fc0c None None] MFA Rules not processed for user `5ce3c93ac7d04f6ea76fd82a2bc810cb`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:50.118298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-baf395d4-952b-4d5d-b69c-511b9f87fc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:50.200262 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1d4813d4-4bd6-4038-8b0f-540c442c2c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01dbefa7fffe4eba9f1b0eb19390735f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:50.209111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1d4813d4-4bd6-4038-8b0f-540c442c2c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01dbefa7fffe4eba9f1b0eb19390735f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:50.217912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1006/1977] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:50.235024 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf395d4-952b-4d5d-b69c-511b9f87fc0c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:50.237096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-baf395d4-952b-4d5d-b69c-511b9f87fc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:50.237848 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1d4813d4-4bd6-4038-8b0f-540c442c2c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01dbefa7fffe4eba9f1b0eb19390735f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:50.249698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 972/1978] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:49 2022] DELETE /identity/v3/users/01dbefa7fffe4eba9f1b0eb19390735f => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:50.260970 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62c36b-8a21-4129-a2ae-361b347c1e3c None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:50.261255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62c36b-8a21-4129-a2ae-361b347c1e3c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:50.261707 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62c36b-8a21-4129-a2ae-361b347c1e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:50.286864 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-8d62c36b-8a21-4129-a2ae-361b347c1e3c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:50.288525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8d62c36b-8a21-4129-a2ae-361b347c1e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:50.356621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf395d4-952b-4d5d-b69c-511b9f87fc0c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:50.394196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 973/1979] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:50.402484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62c36b-8a21-4129-a2ae-361b347c1e3c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:50.403899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8d62c36b-8a21-4129-a2ae-361b347c1e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:50.442287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e971f-e466-401d-8713-c931476861fa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e021a00524341359bc4b5a800a9bbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:50.443071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e971f-e466-401d-8713-c931476861fa service glance] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:50.443426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e971f-e466-401d-8713-c931476861fa service glance] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:50.444241 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e971f-e466-401d-8713-c931476861fa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:50.446667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2c2e971f-e466-401d-8713-c931476861fa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:50.476141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62c36b-8a21-4129-a2ae-361b347c1e3c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:50.480993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9976b7e-bc06-4b33-ab5b-7b6915a6e6f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:50.483465 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9976b7e-bc06-4b33-ab5b-7b6915a6e6f3 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:50.483465 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9976b7e-bc06-4b33-ab5b-7b6915a6e6f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:50.483465 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9976b7e-bc06-4b33-ab5b-7b6915a6e6f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:50.492214 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9976b7e-bc06-4b33-ab5b-7b6915a6e6f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:50.494261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9976b7e-bc06-4b33-ab5b-7b6915a6e6f3 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:50.506291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 974/1980] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:50.522005 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e971f-e466-401d-8713-c931476861fa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:50.523532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e971f-e466-401d-8713-c931476861fa service glance] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:50.534635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1007/1981] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:50.552247 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:50.947110 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4457d1-41d4-4688-9497-caaa9247ab83 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:50.947366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4457d1-41d4-4688-9497-caaa9247ab83 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:50.947667 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4457d1-41d4-4688-9497-caaa9247ab83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:50.967908 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-3b4457d1-41d4-4688-9497-caaa9247ab83 None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:50.969108 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-3b4457d1-41d4-4688-9497-caaa9247ab83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:51.015806 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e971f-e466-401d-8713-c931476861fa service glance] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:51.017874 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2c2e971f-e466-401d-8713-c931476861fa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:51.034905 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 975/1982] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 19:56:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:51.042441 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e971f-e466-401d-8713-c931476861fa service glance] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:51.048998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:51.049732 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:51.050513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:51.050805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] PATH_INFO: `/v3/projects/1aefb4089113410da2d3cc684bc20b02` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:51.051498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4457d1-41d4-4688-9497-caaa9247ab83 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:51.052728 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aefb4089113410da2d3cc684bc20b02)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:51.053286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-3b4457d1-41d4-4688-9497-caaa9247ab83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:51.055143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:51.122808 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4457d1-41d4-4688-9497-caaa9247ab83 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:51.127843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07338915-c4bf-4fbf-b83f-158e01d19496 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:51.128444 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07338915-c4bf-4fbf-b83f-158e01d19496 service swift] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:51.128717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07338915-c4bf-4fbf-b83f-158e01d19496 service swift] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:51.128983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07338915-c4bf-4fbf-b83f-158e01d19496 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:51.136911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07338915-c4bf-4fbf-b83f-158e01d19496 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:51.139546 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07338915-c4bf-4fbf-b83f-158e01d19496 service swift] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:51.147654 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 976/1983] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 19:56:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 19:56:51.302498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aefb4089113410da2d3cc684bc20b02', 'request_id': 'req-cbeba457-0415-43fc-a814-0311529e18d1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:51.394260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aefb4089113410da2d3cc684bc20b02', 'request_id': 'req-cbeba457-0415-43fc-a814-0311529e18d1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:51.406349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aefb4089113410da2d3cc684bc20b02', 'request_id': 'req-cbeba457-0415-43fc-a814-0311529e18d1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:51.428054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] The token cache is being invalidate because project 1aefb4089113410da2d3cc684bc20b02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:51.428349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:51.442962 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:51.442962 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1008/1984] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:51 2022] DELETE /identity/v3/projects/1aefb4089113410da2d3cc684bc20b02 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:51.443724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07338915-c4bf-4fbf-b83f-158e01d19496 service swift] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:51.467352 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07338915-c4bf-4fbf-b83f-158e01d19496 service swift] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:51.558267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee92bb-11bb-44e9-8305-dab15b0efa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:51.559268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee92bb-11bb-44e9-8305-dab15b0efa3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:51.559875 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee92bb-11bb-44e9-8305-dab15b0efa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:51.560326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee92bb-11bb-44e9-8305-dab15b0efa3e admin admin] PATH_INFO: `/v3/users/018194ce21184e24a557c1b9f3de6e5b` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:51.561770 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee92bb-11bb-44e9-8305-dab15b0efa3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=018194ce21184e24a557c1b9f3de6e5b)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:51.564169 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee92bb-11bb-44e9-8305-dab15b0efa3e admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:51.896242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-85ee92bb-11bb-44e9-8305-dab15b0efa3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '018194ce21184e24a557c1b9f3de6e5b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:51.911456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-85ee92bb-11bb-44e9-8305-dab15b0efa3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '018194ce21184e24a557c1b9f3de6e5b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:52.500109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-85ee92bb-11bb-44e9-8305-dab15b0efa3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '018194ce21184e24a557c1b9f3de6e5b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:52.500768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-85ee92bb-11bb-44e9-8305-dab15b0efa3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '018194ce21184e24a557c1b9f3de6e5b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:52.513275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-85ee92bb-11bb-44e9-8305-dab15b0efa3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '018194ce21184e24a557c1b9f3de6e5b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:52.529843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 977/1985] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:51 2022] DELETE /identity/v3/users/018194ce21184e24a557c1b9f3de6e5b => generated 0 bytes in 1091 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:52.562098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:53.257408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:53.257408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:53.285941 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeba457-0415-43fc-a814-0311529e18d1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:53.429713 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:53.430860 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:53.431376 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:53.432201 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] PATH_INFO: `/v3/projects/189d3d2f10bc42f5864055feec04cb4c` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:53.433340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=189d3d2f10bc42f5864055feec04cb4c)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:53.436119 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:53.621438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '189d3d2f10bc42f5864055feec04cb4c', 'request_id': 'req-4bab8786-a262-4507-a6ab-724a78e38e71'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:53.711359 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '189d3d2f10bc42f5864055feec04cb4c', 'request_id': 'req-4bab8786-a262-4507-a6ab-724a78e38e71'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:53.722058 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '189d3d2f10bc42f5864055feec04cb4c', 'request_id': 'req-4bab8786-a262-4507-a6ab-724a78e38e71'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:53.738820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] The token cache is being invalidate because project 189d3d2f10bc42f5864055feec04cb4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:53.739122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:53.739796 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:53.742089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1009/1986] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:53 2022] DELETE /identity/v3/projects/189d3d2f10bc42f5864055feec04cb4c => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:53.747030 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faaa46-b8dd-495c-a55c-785861b0646a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:53.750844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faaa46-b8dd-495c-a55c-785861b0646a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:53.750844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52faaa46-b8dd-495c-a55c-785861b0646a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:53.775096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-52faaa46-b8dd-495c-a55c-785861b0646a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:53.776319 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-52faaa46-b8dd-495c-a55c-785861b0646a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:53.822736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:53.824603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:53.893508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 978/1987] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:53.900220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910ca53-e42c-4b38-9b23-1a16d430e28f None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:53.900507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910ca53-e42c-4b38-9b23-1a16d430e28f None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:53.900780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910ca53-e42c-4b38-9b23-1a16d430e28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:53.937685 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-4910ca53-e42c-4b38-9b23-1a16d430e28f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:53.940510 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4910ca53-e42c-4b38-9b23-1a16d430e28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:53.949604 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 979/1988] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:53.954023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910ca53-e42c-4b38-9b23-1a16d430e28f None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:53.967079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910ca53-e42c-4b38-9b23-1a16d430e28f None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:53.972645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9111bb2-ed95-4980-8664-567073eeff63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:53.993949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9111bb2-ed95-4980-8664-567073eeff63 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:53.993949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9111bb2-ed95-4980-8664-567073eeff63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:53.993949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9111bb2-ed95-4980-8664-567073eeff63 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:53.993949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9111bb2-ed95-4980-8664-567073eeff63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:53.993949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9111bb2-ed95-4980-8664-567073eeff63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:53.993949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9111bb2-ed95-4980-8664-567073eeff63 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:54.000717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 980/1989] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:56:54.005398 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9111bb2-ed95-4980-8664-567073eeff63 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:54.008705 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab8786-a262-4507-a6ab-724a78e38e71 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:54.018950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178e95b-cff4-401a-b546-3207e1860dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:54.021599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178e95b-cff4-401a-b546-3207e1860dce admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:54.021599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178e95b-cff4-401a-b546-3207e1860dce admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:54.021599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178e95b-cff4-401a-b546-3207e1860dce admin admin] PATH_INFO: `/v3/users/088fc1d4dee84e6391c457c84d4506b9` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:54.022858 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178e95b-cff4-401a-b546-3207e1860dce admin admin] RBAC: Authorizing `identity:delete_user(user_id=088fc1d4dee84e6391c457c84d4506b9)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:54.024175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9111bb2-ed95-4980-8664-567073eeff63 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:54.027749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178e95b-cff4-401a-b546-3207e1860dce admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:54.033958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b8fd6-af68-41dc-82fd-5502f2cecff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:54.033958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b8fd6-af68-41dc-82fd-5502f2cecff0 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:54.033958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b8fd6-af68-41dc-82fd-5502f2cecff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:54.033958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b8fd6-af68-41dc-82fd-5502f2cecff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:54.034652 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b8fd6-af68-41dc-82fd-5502f2cecff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:54.035682 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b8fd6-af68-41dc-82fd-5502f2cecff0 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:54.240542 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 981/1990] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:54 2022] POST /identity/v3/projects => generated 385 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:54.242107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b8fd6-af68-41dc-82fd-5502f2cecff0 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:54.379124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6178e95b-cff4-401a-b546-3207e1860dce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '088fc1d4dee84e6391c457c84d4506b9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:54.452015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b8fd6-af68-41dc-82fd-5502f2cecff0 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:54.459172 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fc045-1747-47b0-abd1-8adde1cba16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:54.459807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fc045-1747-47b0-abd1-8adde1cba16a admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:54.460093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fc045-1747-47b0-abd1-8adde1cba16a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:54.460366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281fc045-1747-47b0-abd1-8adde1cba16a admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:54.461004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fc045-1747-47b0-abd1-8adde1cba16a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:54.463786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281fc045-1747-47b0-abd1-8adde1cba16a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:54.558873 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6178e95b-cff4-401a-b546-3207e1860dce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '088fc1d4dee84e6391c457c84d4506b9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:54.562639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:54.584617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6178e95b-cff4-401a-b546-3207e1860dce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '088fc1d4dee84e6391c457c84d4506b9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:54.585995 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6178e95b-cff4-401a-b546-3207e1860dce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '088fc1d4dee84e6391c457c84d4506b9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:54.587179 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6178e95b-cff4-401a-b546-3207e1860dce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '088fc1d4dee84e6391c457c84d4506b9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:54.620652 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1010/1991] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:56:53 2022] DELETE /identity/v3/users/088fc1d4dee84e6391c457c84d4506b9 => generated 0 bytes in 799 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:54.625113 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd419b-7e11-408e-872c-5bf1f48444ce None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:54.625474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd419b-7e11-408e-872c-5bf1f48444ce None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:54.625863 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd419b-7e11-408e-872c-5bf1f48444ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:54.663937 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 982/1992] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:54 2022] POST /identity/v3/users => generated 293 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:54.666853 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fc045-1747-47b0-abd1-8adde1cba16a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:54.668683 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-281fc045-1747-47b0-abd1-8adde1cba16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:54.674854 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bcfd419b-7e11-408e-872c-5bf1f48444ce None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:54.676270 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bcfd419b-7e11-408e-872c-5bf1f48444ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:54.718323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281fc045-1747-47b0-abd1-8adde1cba16a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:54.773639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1011/1993] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:54.792453 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd419b-7e11-408e-872c-5bf1f48444ce None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:54.795246 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bcfd419b-7e11-408e-872c-5bf1f48444ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:54.949918 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1f2b3-9c7f-4271-8f87-f58c70b43cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:54.952430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1f2b3-9c7f-4271-8f87-f58c70b43cde admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:54.952430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1f2b3-9c7f-4271-8f87-f58c70b43cde admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:54.952430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1f2b3-9c7f-4271-8f87-f58c70b43cde admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:54.952430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1f2b3-9c7f-4271-8f87-f58c70b43cde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:54.954374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1f2b3-9c7f-4271-8f87-f58c70b43cde admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:54.967375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd419b-7e11-408e-872c-5bf1f48444ce None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:54.982061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 983/1994] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:54.982061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1f2b3-9c7f-4271-8f87-f58c70b43cde admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:54.982290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a43a09-8b03-42b6-a79e-b945d89b4f0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:54.982290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a43a09-8b03-42b6-a79e-b945d89b4f0c service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:54.982991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a43a09-8b03-42b6-a79e-b945d89b4f0c service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:54.983275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a43a09-8b03-42b6-a79e-b945d89b4f0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:54.996326 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a43a09-8b03-42b6-a79e-b945d89b4f0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:54.997896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a43a09-8b03-42b6-a79e-b945d89b4f0c service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:55.001540 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1f2b3-9c7f-4271-8f87-f58c70b43cde admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:55.014605 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28773913-e635-4207-ab78-bf8e4416eb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:55.015240 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28773913-e635-4207-ab78-bf8e4416eb1d admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:55.015521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28773913-e635-4207-ab78-bf8e4416eb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:55.015798 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28773913-e635-4207-ab78-bf8e4416eb1d admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:55.017190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1012/1995] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:56:55.019149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28773913-e635-4207-ab78-bf8e4416eb1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:55.026636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28773913-e635-4207-ab78-bf8e4416eb1d admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:55.046287 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 984/1996] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:54 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:55.052874 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a43a09-8b03-42b6-a79e-b945d89b4f0c service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:55.070377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a43a09-8b03-42b6-a79e-b945d89b4f0c service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:55.118884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1781a-bba1-4451-9ee8-81331d292891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:55.118884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1781a-bba1-4451-9ee8-81331d292891 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:55.118884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1781a-bba1-4451-9ee8-81331d292891 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:55.118884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1781a-bba1-4451-9ee8-81331d292891 admin admin] PATH_INFO: `/v3/projects/756419ab2bc64870a61c623ba2316e1d/users/7bc8055aeedd44b1883652a877200f22/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:55.182899 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1781a-bba1-4451-9ee8-81331d292891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=756419ab2bc64870a61c623ba2316e1d, user_id=7bc8055aeedd44b1883652a877200f22, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:55.193617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1781a-bba1-4451-9ee8-81331d292891 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:55.441007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1013/1997] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:55 2022] PUT /identity/v3/projects/756419ab2bc64870a61c623ba2316e1d/users/7bc8055aeedd44b1883652a877200f22/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:55.452791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbab542-64a7-4ea7-8418-a1401505bd49 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:55.453023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbab542-64a7-4ea7-8418-a1401505bd49 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:55.453281 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbab542-64a7-4ea7-8418-a1401505bd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:55.484244 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-0cbab542-64a7-4ea7-8418-a1401505bd49 None None] MFA Rules not processed for user `7bc8055aeedd44b1883652a877200f22`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:55.485539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0cbab542-64a7-4ea7-8418-a1401505bd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:55.567892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 985/1998] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:55.572685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1781a-bba1-4451-9ee8-81331d292891 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:55.643258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1781a-bba1-4451-9ee8-81331d292891 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:55.649553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05934f53-815b-42c8-b1a7-76e8dc14e7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:55.650205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05934f53-815b-42c8-b1a7-76e8dc14e7be admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:55.650481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05934f53-815b-42c8-b1a7-76e8dc14e7be admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:55.650775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05934f53-815b-42c8-b1a7-76e8dc14e7be admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:55.651410 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05934f53-815b-42c8-b1a7-76e8dc14e7be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:55.653332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05934f53-815b-42c8-b1a7-76e8dc14e7be admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:55.819731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1014/1999] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:55 2022] POST /identity/v3/projects => generated 385 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:56:55.843435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbab542-64a7-4ea7-8418-a1401505bd49 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:55.867225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbab542-64a7-4ea7-8418-a1401505bd49 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:55.973663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb82623-49dc-475e-aa92-33ba51d45882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:55.974597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb82623-49dc-475e-aa92-33ba51d45882 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:55.975149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb82623-49dc-475e-aa92-33ba51d45882 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:55.975545 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb82623-49dc-475e-aa92-33ba51d45882 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:55.976293 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb82623-49dc-475e-aa92-33ba51d45882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:55.980729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb82623-49dc-475e-aa92-33ba51d45882 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:56.085490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 986/2000] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:55 2022] POST /identity/v3/users => generated 299 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:56.086844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05934f53-815b-42c8-b1a7-76e8dc14e7be admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:56.101091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05934f53-815b-42c8-b1a7-76e8dc14e7be admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:56.104262 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:DELETE=9,keystone:SELECT=85,keystone:INSERT=6 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:56.106561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3af0f3-ffce-4d00-a288-ce7937535574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:56.109135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3af0f3-ffce-4d00-a288-ce7937535574 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:56.109422 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3af0f3-ffce-4d00-a288-ce7937535574 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:56.109702 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3af0f3-ffce-4d00-a288-ce7937535574 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:56.110425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3af0f3-ffce-4d00-a288-ce7937535574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:56.112599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3af0f3-ffce-4d00-a288-ce7937535574 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:56.142205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1015/2001] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:56.143069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb82623-49dc-475e-aa92-33ba51d45882 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:56.156342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb82623-49dc-475e-aa92-33ba51d45882 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:56.165922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2843d4-c0c5-4a82-b216-db1b4c7cfbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:56.166714 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3af0f3-ffce-4d00-a288-ce7937535574 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:56.167188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2843d4-c0c5-4a82-b216-db1b4c7cfbab admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:56.167572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2843d4-c0c5-4a82-b216-db1b4c7cfbab admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:56.167940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2843d4-c0c5-4a82-b216-db1b4c7cfbab admin admin] PATH_INFO: `/v3/projects/008134341efb416faeffabc5c3dda2c3/users/be78381048f84b34b99ebc2bc6961b23/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:56.168429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-af3af0f3-ffce-4d00-a288-ce7937535574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:56.189233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3af0f3-ffce-4d00-a288-ce7937535574 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:56.199200 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:56.199859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:56.200138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:56.200422 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] PATH_INFO: `/v3/projects/4b300df70349459ca3134366ca97b134` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:56.201620 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b300df70349459ca3134366ca97b134)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:56.201895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2843d4-c0c5-4a82-b216-db1b4c7cfbab admin admin] RBAC: Authorizing `identity:create_grant(project_id=008134341efb416faeffabc5c3dda2c3, user_id=be78381048f84b34b99ebc2bc6961b23, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:56.207066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2843d4-c0c5-4a82-b216-db1b4c7cfbab admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:56.214613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:56.293569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 987/2002] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:56 2022] PUT /identity/v3/projects/008134341efb416faeffabc5c3dda2c3/users/be78381048f84b34b99ebc2bc6961b23/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:56:56.297144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2843d4-c0c5-4a82-b216-db1b4c7cfbab admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:56.370283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b300df70349459ca3134366ca97b134', 'request_id': 'req-d033be03-eb81-48db-8155-e2cb2257f137'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:56.457874 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2843d4-c0c5-4a82-b216-db1b4c7cfbab admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:56.517441 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b300df70349459ca3134366ca97b134', 'request_id': 'req-d033be03-eb81-48db-8155-e2cb2257f137'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:56.529327 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b300df70349459ca3134366ca97b134', 'request_id': 'req-d033be03-eb81-48db-8155-e2cb2257f137'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:56.536155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6b672-9f04-4270-acc9-ebcc329ce114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:56.536990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6b672-9f04-4270-acc9-ebcc329ce114 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:56.537286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6b672-9f04-4270-acc9-ebcc329ce114 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:56.537548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6b672-9f04-4270-acc9-ebcc329ce114 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:56.538433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6b672-9f04-4270-acc9-ebcc329ce114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:56.540464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6b672-9f04-4270-acc9-ebcc329ce114 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:56.552133 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] The token cache is being invalidate because project 4b300df70349459ca3134366ca97b134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:56:56.552520 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:56.554185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:56:56.557068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1016/2004] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:56:56 2022] DELETE /identity/v3/projects/4b300df70349459ca3134366ca97b134 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:56.559384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:56.561544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 988/2004] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:56 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:56:56.562242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:56.566478 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb87089-202f-4d26-81de-a9f03453cae8 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:56.566478 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb87089-202f-4d26-81de-a9f03453cae8 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:56.566478 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb87089-202f-4d26-81de-a9f03453cae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:56.587824 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-3cb87089-202f-4d26-81de-a9f03453cae8 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:56.589152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-3cb87089-202f-4d26-81de-a9f03453cae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:56.591907 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033be03-eb81-48db-8155-e2cb2257f137 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:56.600030 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c4355-3532-4e47-b1dc-baacb6bb7a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:56.601062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 989/2005] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:56.602294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c4355-3532-4e47-b1dc-baacb6bb7a04 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:56.602948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c4355-3532-4e47-b1dc-baacb6bb7a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:56.603429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c4355-3532-4e47-b1dc-baacb6bb7a04 admin admin] PATH_INFO: `/v3/projects/008134341efb416faeffabc5c3dda2c3/users/be78381048f84b34b99ebc2bc6961b23/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:56.605613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c4355-3532-4e47-b1dc-baacb6bb7a04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=008134341efb416faeffabc5c3dda2c3, user_id=be78381048f84b34b99ebc2bc6961b23, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:56.610467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c4355-3532-4e47-b1dc-baacb6bb7a04 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:56.612756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8ecc3-ad53-429a-af62-9d4b9ab1ca29 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:56.613044 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8ecc3-ad53-429a-af62-9d4b9ab1ca29 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:56.613320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b8ecc3-ad53-429a-af62-9d4b9ab1ca29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:56.629875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1017/2006] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:56 2022] PUT /identity/v3/projects/008134341efb416faeffabc5c3dda2c3/users/be78381048f84b34b99ebc2bc6961b23/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:56.640913 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd8345-8cda-4e32-92a7-28928eed33a5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:56.641342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd8345-8cda-4e32-92a7-28928eed33a5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:56.641726 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd8345-8cda-4e32-92a7-28928eed33a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:56.647638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-c3b8ecc3-ad53-429a-af62-9d4b9ab1ca29 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:56.650452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c3b8ecc3-ad53-429a-af62-9d4b9ab1ca29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:56.675730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-76fd8345-8cda-4e32-92a7-28928eed33a5 None None] MFA Rules not processed for user `be78381048f84b34b99ebc2bc6961b23`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:56.678095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-76fd8345-8cda-4e32-92a7-28928eed33a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:56.735378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 990/2007] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:56.737427 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8ecc3-ad53-429a-af62-9d4b9ab1ca29 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:56.747389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b8ecc3-ad53-429a-af62-9d4b9ab1ca29 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:56.753709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c130c4-d398-48fc-9499-ffed6375c824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:56.754360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c130c4-d398-48fc-9499-ffed6375c824 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:56.754640 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c130c4-d398-48fc-9499-ffed6375c824 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:56.754910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c130c4-d398-48fc-9499-ffed6375c824 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:56.755600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c130c4-d398-48fc-9499-ffed6375c824 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:56:56.755906 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c130c4-d398-48fc-9499-ffed6375c824 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:56.757501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c130c4-d398-48fc-9499-ffed6375c824 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:56.771926 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1018/2008] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:56:56.776613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 991/2009] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:56:56.778852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd8345-8cda-4e32-92a7-28928eed33a5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:56.784898 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c130c4-d398-48fc-9499-ffed6375c824 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:56.791661 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-81c130c4-d398-48fc-9499-ffed6375c824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:56.791816 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd8345-8cda-4e32-92a7-28928eed33a5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:56.798076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd112c-5e6c-46e4-93af-28454a06485c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:56.798814 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd112c-5e6c-46e4-93af-28454a06485c admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:56.799802 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd112c-5e6c-46e4-93af-28454a06485c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:56.800141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd112c-5e6c-46e4-93af-28454a06485c admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:56.800839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd112c-5e6c-46e4-93af-28454a06485c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:56.802889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd112c-5e6c-46e4-93af-28454a06485c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:56.921135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c130c4-d398-48fc-9499-ffed6375c824 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:56.936979 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1019/2010] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:56:56 2022] POST /identity/v3/projects => generated 367 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:56.940654 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509fcb2-56b6-4e7b-9a00-7eb1ed895b22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:56.942433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd112c-5e6c-46e4-93af-28454a06485c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:56.942892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509fcb2-56b6-4e7b-9a00-7eb1ed895b22 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:56.943176 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509fcb2-56b6-4e7b-9a00-7eb1ed895b22 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:56.943453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509fcb2-56b6-4e7b-9a00-7eb1ed895b22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:56.948791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4509fcb2-56b6-4e7b-9a00-7eb1ed895b22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:57.074042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509fcb2-56b6-4e7b-9a00-7eb1ed895b22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:57.074042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509fcb2-56b6-4e7b-9a00-7eb1ed895b22 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:57.075071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd112c-5e6c-46e4-93af-28454a06485c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:57.081269 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341caf6e-b23c-4c69-be89-c04ed5441317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:57.082010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341caf6e-b23c-4c69-be89-c04ed5441317 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:57.082356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341caf6e-b23c-4c69-be89-c04ed5441317 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:57.082697 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341caf6e-b23c-4c69-be89-c04ed5441317 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:57.083629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341caf6e-b23c-4c69-be89-c04ed5441317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:57.085979 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341caf6e-b23c-4c69-be89-c04ed5441317 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:57.094004 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 992/2011] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:57.099128 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:57.252921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1020/2012] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:56:56 2022] POST /identity/v3/users => generated 284 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:56:57.255915 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509fcb2-56b6-4e7b-9a00-7eb1ed895b22 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:57.270954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509fcb2-56b6-4e7b-9a00-7eb1ed895b22 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:57.279726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e226f00-786a-4bfd-a2a7-cfbfbac870e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:57.280353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e226f00-786a-4bfd-a2a7-cfbfbac870e4 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:57.280633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e226f00-786a-4bfd-a2a7-cfbfbac870e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:57.280910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e226f00-786a-4bfd-a2a7-cfbfbac870e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:57.281550 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e226f00-786a-4bfd-a2a7-cfbfbac870e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:57.283242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e226f00-786a-4bfd-a2a7-cfbfbac870e4 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:57.297963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 993/2013] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:57.300176 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341caf6e-b23c-4c69-be89-c04ed5441317 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:57.316613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341caf6e-b23c-4c69-be89-c04ed5441317 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:57.327078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855e2ef-4ca2-4758-8ab7-11adb32a2305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:57.327991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855e2ef-4ca2-4758-8ab7-11adb32a2305 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:57.328355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855e2ef-4ca2-4758-8ab7-11adb32a2305 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:57.328682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855e2ef-4ca2-4758-8ab7-11adb32a2305 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:57.329311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855e2ef-4ca2-4758-8ab7-11adb32a2305 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:57.330989 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855e2ef-4ca2-4758-8ab7-11adb32a2305 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:57.345500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1021/2014] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:56:57.349602 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e226f00-786a-4bfd-a2a7-cfbfbac870e4 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:57.363095 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e226f00-786a-4bfd-a2a7-cfbfbac870e4 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:57.368767 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eff981-0dd5-4719-8d13-18c6db4a93e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:57.369673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eff981-0dd5-4719-8d13-18c6db4a93e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:57.370078 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eff981-0dd5-4719-8d13-18c6db4a93e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:57.370489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eff981-0dd5-4719-8d13-18c6db4a93e9 admin admin] PATH_INFO: `/v3/projects/d1ec7e8608044db896133a6b38946299/users/8136dabd3c184f16873c64459f4ad312/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:57.391396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eff981-0dd5-4719-8d13-18c6db4a93e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1ec7e8608044db896133a6b38946299, user_id=8136dabd3c184f16873c64459f4ad312, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:57.395730 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eff981-0dd5-4719-8d13-18c6db4a93e9 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:57.507128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 994/2015] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:56:57 2022] PUT /identity/v3/projects/d1ec7e8608044db896133a6b38946299/users/8136dabd3c184f16873c64459f4ad312/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:56:57.518206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff166e4e-2c7b-41cc-9f3f-104e18196b1f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:57.518698 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff166e4e-2c7b-41cc-9f3f-104e18196b1f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:57.518964 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff166e4e-2c7b-41cc-9f3f-104e18196b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:57.552608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-ff166e4e-2c7b-41cc-9f3f-104e18196b1f None None] MFA Rules not processed for user `8136dabd3c184f16873c64459f4ad312`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:56:57.554350 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ff166e4e-2c7b-41cc-9f3f-104e18196b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:57.633157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1022/2016] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:56:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:56:57.647137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eff981-0dd5-4719-8d13-18c6db4a93e9 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:56:57.649097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-12eff981-0dd5-4719-8d13-18c6db4a93e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:56:57.724945 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eff981-0dd5-4719-8d13-18c6db4a93e9 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:56:57.729383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ddb22-b12e-4baf-a171-63f0922d6767 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e021a00524341359bc4b5a800a9bbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:56:57.730015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ddb22-b12e-4baf-a171-63f0922d6767 service glance] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:56:57.730294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ddb22-b12e-4baf-a171-63f0922d6767 service glance] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:56:57.730591 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ddb22-b12e-4baf-a171-63f0922d6767 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:56:57.742696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ddb22-b12e-4baf-a171-63f0922d6767 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:56:57.744163 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ddb22-b12e-4baf-a171-63f0922d6767 service glance] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:56:57.761984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 995/2017] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:56:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:56:58.075861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:56:59.089517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:00.573684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:01.671819 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:02.340310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:04.677010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff166e4e-2c7b-41cc-9f3f-104e18196b1f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:04.767491 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff166e4e-2c7b-41cc-9f3f-104e18196b1f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:04.771765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daaf2cf-d7f1-4954-a55c-bc2f385b11a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:04.772376 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daaf2cf-d7f1-4954-a55c-bc2f385b11a0 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:04.772675 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daaf2cf-d7f1-4954-a55c-bc2f385b11a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:04.772960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daaf2cf-d7f1-4954-a55c-bc2f385b11a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:04.853572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daaf2cf-d7f1-4954-a55c-bc2f385b11a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:04.853572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daaf2cf-d7f1-4954-a55c-bc2f385b11a0 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:04.860925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b08cb-4e2a-4705-9ef4-2b76945d5ac5 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:04.862025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b08cb-4e2a-4705-9ef4-2b76945d5ac5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:04.862025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35b08cb-4e2a-4705-9ef4-2b76945d5ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:04.866383 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1023/2018] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:04 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:04.896298 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b35b08cb-4e2a-4705-9ef4-2b76945d5ac5 None None] MFA Rules not processed for user `0ebafc08399c4853959800883742eb78`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:04.897585 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b35b08cb-4e2a-4705-9ef4-2b76945d5ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:04.911804 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:04.970040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 996/2019] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:05.218344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:05.396241 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daaf2cf-d7f1-4954-a55c-bc2f385b11a0 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:05.397830 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3daaf2cf-d7f1-4954-a55c-bc2f385b11a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:05.484959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daaf2cf-d7f1-4954-a55c-bc2f385b11a0 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:05.490426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500348e9-d5ce-4435-8869-d47f7cc51e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:05.491077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500348e9-d5ce-4435-8869-d47f7cc51e1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:05.491390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500348e9-d5ce-4435-8869-d47f7cc51e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:05.491665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500348e9-d5ce-4435-8869-d47f7cc51e1a admin admin] PATH_INFO: `/v3/users/7ed1e55827d041bf872dd067e1bea402` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:05.492861 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500348e9-d5ce-4435-8869-d47f7cc51e1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed1e55827d041bf872dd067e1bea402)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:05.494792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500348e9-d5ce-4435-8869-d47f7cc51e1a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:05.740505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-500348e9-d5ce-4435-8869-d47f7cc51e1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed1e55827d041bf872dd067e1bea402'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:05.917434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-500348e9-d5ce-4435-8869-d47f7cc51e1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed1e55827d041bf872dd067e1bea402'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:05.931614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-500348e9-d5ce-4435-8869-d47f7cc51e1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed1e55827d041bf872dd067e1bea402'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:05.932433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-500348e9-d5ce-4435-8869-d47f7cc51e1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed1e55827d041bf872dd067e1bea402'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:05.933098 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-500348e9-d5ce-4435-8869-d47f7cc51e1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed1e55827d041bf872dd067e1bea402'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:05.946372 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1024/2020] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:57:05 2022] DELETE /identity/v3/users/7ed1e55827d041bf872dd067e1bea402 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:05.951143 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791182c-f499-45b4-904b-677956348d2d None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:05.951595 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791182c-f499-45b4-904b-677956348d2d None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:05.951922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791182c-f499-45b4-904b-677956348d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:05.972310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-5791182c-f499-45b4-904b-677956348d2d None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:05.973558 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-5791182c-f499-45b4-904b-677956348d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:06.054195 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 997/2021] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:06.095203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500348e9-d5ce-4435-8869-d47f7cc51e1a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:06.096338 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-500348e9-d5ce-4435-8869-d47f7cc51e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:06.225381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500348e9-d5ce-4435-8869-d47f7cc51e1a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:06.225381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3da26-e3be-449c-bdc1-b9bd4ffff379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:06.225381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3da26-e3be-449c-bdc1-b9bd4ffff379 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:06.225381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3da26-e3be-449c-bdc1-b9bd4ffff379 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:06.226755 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3da26-e3be-449c-bdc1-b9bd4ffff379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:06.237340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3da26-e3be-449c-bdc1-b9bd4ffff379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:06.238944 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3da26-e3be-449c-bdc1-b9bd4ffff379 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:06.257998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1025/2022] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:06.421614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:06.959970 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:07.017471 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791182c-f499-45b4-904b-677956348d2d None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:07.020179 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-5791182c-f499-45b4-904b-677956348d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:07.033398 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791182c-f499-45b4-904b-677956348d2d None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:07.047038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:07.047705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:07.048461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:07.048807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] PATH_INFO: `/v3/projects/8b20d487ab6246bbb42631a5dcf5d16c` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:07.050295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b20d487ab6246bbb42631a5dcf5d16c)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:07.052331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:07.094939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3da26-e3be-449c-bdc1-b9bd4ffff379 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:07.098074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-17f3da26-e3be-449c-bdc1-b9bd4ffff379 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:07.110827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3da26-e3be-449c-bdc1-b9bd4ffff379 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:07.116034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0ec57-d70e-4687-8617-29246171f0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:07.116977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0ec57-d70e-4687-8617-29246171f0fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:07.116977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0ec57-d70e-4687-8617-29246171f0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:07.117183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0ec57-d70e-4687-8617-29246171f0fb admin admin] PATH_INFO: `/v3/users/96428b8e76314536a9e7033a841454e2` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:07.118322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c0ec57-d70e-4687-8617-29246171f0fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=96428b8e76314536a9e7033a841454e2)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:07.120314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c0ec57-d70e-4687-8617-29246171f0fb admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:07.182850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b20d487ab6246bbb42631a5dcf5d16c', 'request_id': 'req-828890c2-637f-4213-8d7f-0472894ef64e'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.188381 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b20d487ab6246bbb42631a5dcf5d16c', 'request_id': 'req-828890c2-637f-4213-8d7f-0472894ef64e'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.203255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b20d487ab6246bbb42631a5dcf5d16c', 'request_id': 'req-828890c2-637f-4213-8d7f-0472894ef64e'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.287255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] The token cache is being invalidate because project 8b20d487ab6246bbb42631a5dcf5d16c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:57:07.287255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.287255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.287255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 998/2023] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:57:07 2022] DELETE /identity/v3/projects/8b20d487ab6246bbb42631a5dcf5d16c => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:07.289327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:07.290776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:07.354801 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828890c2-637f-4213-8d7f-0472894ef64e admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:07.359542 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cc9ec-2040-4c88-8973-51575a2a0df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:07.360187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cc9ec-2040-4c88-8973-51575a2a0df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:07.360478 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cc9ec-2040-4c88-8973-51575a2a0df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:07.360751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2cc9ec-2040-4c88-8973-51575a2a0df3 admin admin] PATH_INFO: `/v3/users/6ccb9e0ad9a94525a6a4a2a1041349ba` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:07.362473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cc9ec-2040-4c88-8973-51575a2a0df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ccb9e0ad9a94525a6a4a2a1041349ba)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:07.364390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2cc9ec-2040-4c88-8973-51575a2a0df3 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:07.389379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-a4c0ec57-d70e-4687-8617-29246171f0fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96428b8e76314536a9e7033a841454e2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.520801 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-a4c0ec57-d70e-4687-8617-29246171f0fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96428b8e76314536a9e7033a841454e2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.533268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-a4c0ec57-d70e-4687-8617-29246171f0fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96428b8e76314536a9e7033a841454e2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.534452 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-a4c0ec57-d70e-4687-8617-29246171f0fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96428b8e76314536a9e7033a841454e2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.535052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-a4c0ec57-d70e-4687-8617-29246171f0fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96428b8e76314536a9e7033a841454e2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.547412 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1026/2024] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:57:07 2022] DELETE /identity/v3/users/96428b8e76314536a9e7033a841454e2 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:07.637578 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ec2cc9ec-2040-4c88-8973-51575a2a0df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ccb9e0ad9a94525a6a4a2a1041349ba'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.650073 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ec2cc9ec-2040-4c88-8973-51575a2a0df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ccb9e0ad9a94525a6a4a2a1041349ba'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.728724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ec2cc9ec-2040-4c88-8973-51575a2a0df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ccb9e0ad9a94525a6a4a2a1041349ba'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.729290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ec2cc9ec-2040-4c88-8973-51575a2a0df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ccb9e0ad9a94525a6a4a2a1041349ba'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.736816 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ec2cc9ec-2040-4c88-8973-51575a2a0df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ccb9e0ad9a94525a6a4a2a1041349ba'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:07.742913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 999/2025] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:57:07 2022] DELETE /identity/v3/users/6ccb9e0ad9a94525a6a4a2a1041349ba => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:57:07.875151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0ec57-d70e-4687-8617-29246171f0fb admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:07.875971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a4c0ec57-d70e-4687-8617-29246171f0fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:07.945342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0ec57-d70e-4687-8617-29246171f0fb admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:08.003397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88479a58-6f95-4f49-b814-619d1f52d51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:08.003966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88479a58-6f95-4f49-b814-619d1f52d51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:08.004366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88479a58-6f95-4f49-b814-619d1f52d51a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:08.004643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88479a58-6f95-4f49-b814-619d1f52d51a admin admin] PATH_INFO: `/v3/projects/b04e9cb110f049d69a2d2747c74c8b51` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:08.005817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88479a58-6f95-4f49-b814-619d1f52d51a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b04e9cb110f049d69a2d2747c74c8b51)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:08.007837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88479a58-6f95-4f49-b814-619d1f52d51a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:08.141944 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-88479a58-6f95-4f49-b814-619d1f52d51a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b04e9cb110f049d69a2d2747c74c8b51', 'request_id': 'req-88479a58-6f95-4f49-b814-619d1f52d51a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:08.253286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cc9ec-2040-4c88-8973-51575a2a0df3 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:08.253286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ec2cc9ec-2040-4c88-8973-51575a2a0df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:08.276547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:08.277405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-88479a58-6f95-4f49-b814-619d1f52d51a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b04e9cb110f049d69a2d2747c74c8b51', 'request_id': 'req-88479a58-6f95-4f49-b814-619d1f52d51a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:08.281623 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2cc9ec-2040-4c88-8973-51575a2a0df3 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:08.289106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-88479a58-6f95-4f49-b814-619d1f52d51a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b04e9cb110f049d69a2d2747c74c8b51', 'request_id': 'req-88479a58-6f95-4f49-b814-619d1f52d51a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:08.302955 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-88479a58-6f95-4f49-b814-619d1f52d51a admin admin] The token cache is being invalidate because project b04e9cb110f049d69a2d2747c74c8b51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:57:08.303392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-88479a58-6f95-4f49-b814-619d1f52d51a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:08.304087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-88479a58-6f95-4f49-b814-619d1f52d51a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:08.306764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1027/2026] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:57:07 2022] DELETE /identity/v3/projects/b04e9cb110f049d69a2d2747c74c8b51 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:08.311855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e937c-0b0d-4fcc-a4a4-15b8aa23613b None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:08.312321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e937c-0b0d-4fcc-a4a4-15b8aa23613b None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:08.316576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e937c-0b0d-4fcc-a4a4-15b8aa23613b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:08.336028 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6c5e937c-0b0d-4fcc-a4a4-15b8aa23613b None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:08.337778 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6c5e937c-0b0d-4fcc-a4a4-15b8aa23613b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:08.387654 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78103237-748e-4e98-8b6b-3dedaced3760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:08.388305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78103237-748e-4e98-8b6b-3dedaced3760 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:08.388692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78103237-748e-4e98-8b6b-3dedaced3760 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:08.388970 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78103237-748e-4e98-8b6b-3dedaced3760 admin admin] PATH_INFO: `/v3/projects/34cb4dd0716d4035919e84c750ad0ccb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:08.390126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78103237-748e-4e98-8b6b-3dedaced3760 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34cb4dd0716d4035919e84c750ad0ccb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:08.392075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78103237-748e-4e98-8b6b-3dedaced3760 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:08.410131 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1028/2027] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:08.417990 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5af038-cee9-46f4-a58c-22b3e66c7c5f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:08.419947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5af038-cee9-46f4-a58c-22b3e66c7c5f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:08.419947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5af038-cee9-46f4-a58c-22b3e66c7c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:08.438879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-9b5af038-cee9-46f4-a58c-22b3e66c7c5f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:08.440781 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9b5af038-cee9-46f4-a58c-22b3e66c7c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:08.449243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1029/2028] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:57:08.453393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5af038-cee9-46f4-a58c-22b3e66c7c5f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:08.463101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5af038-cee9-46f4-a58c-22b3e66c7c5f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:08.468578 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf79c9-90fa-4923-8308-df76abc530f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:08.469201 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf79c9-90fa-4923-8308-df76abc530f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:08.473984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf79c9-90fa-4923-8308-df76abc530f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:08.473984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf79c9-90fa-4923-8308-df76abc530f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:08.473984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf79c9-90fa-4923-8308-df76abc530f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:57:08.473984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf79c9-90fa-4923-8308-df76abc530f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:08.473984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf79c9-90fa-4923-8308-df76abc530f5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:08.484501 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1030/2029] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:57:08.489196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf79c9-90fa-4923-8308-df76abc530f5 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:08.501465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf79c9-90fa-4923-8308-df76abc530f5 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:08.510031 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615cd25-9dff-4f99-a952-d12d2c70df05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:08.510576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615cd25-9dff-4f99-a952-d12d2c70df05 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:08.511014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615cd25-9dff-4f99-a952-d12d2c70df05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:08.511282 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615cd25-9dff-4f99-a952-d12d2c70df05 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:08.511899 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615cd25-9dff-4f99-a952-d12d2c70df05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:08.514146 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615cd25-9dff-4f99-a952-d12d2c70df05 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:08.520411 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-78103237-748e-4e98-8b6b-3dedaced3760 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34cb4dd0716d4035919e84c750ad0ccb', 'request_id': 'req-78103237-748e-4e98-8b6b-3dedaced3760'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:08.539849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-78103237-748e-4e98-8b6b-3dedaced3760 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34cb4dd0716d4035919e84c750ad0ccb', 'request_id': 'req-78103237-748e-4e98-8b6b-3dedaced3760'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:08.546265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-78103237-748e-4e98-8b6b-3dedaced3760 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34cb4dd0716d4035919e84c750ad0ccb', 'request_id': 'req-78103237-748e-4e98-8b6b-3dedaced3760'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:08.623307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-78103237-748e-4e98-8b6b-3dedaced3760 admin admin] The token cache is being invalidate because project 34cb4dd0716d4035919e84c750ad0ccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:57:08.623687 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-78103237-748e-4e98-8b6b-3dedaced3760 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:08.624374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-78103237-748e-4e98-8b6b-3dedaced3760 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:08.626014 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1000/2030] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:57:08 2022] DELETE /identity/v3/projects/34cb4dd0716d4035919e84c750ad0ccb => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:57:08.630506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1031/2031] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:57:08 2022] POST /identity/v3/projects => generated 381 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:57:08.631481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f6500-db14-4ce3-b6a1-b75e585085d7 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:08.631622 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615cd25-9dff-4f99-a952-d12d2c70df05 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:08.631763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f6500-db14-4ce3-b6a1-b75e585085d7 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:08.632033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f6500-db14-4ce3-b6a1-b75e585085d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:08.633132 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5615cd25-9dff-4f99-a952-d12d2c70df05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:08.653196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-a48f6500-db14-4ce3-b6a1-b75e585085d7 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:08.656295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a48f6500-db14-4ce3-b6a1-b75e585085d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:08.701016 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615cd25-9dff-4f99-a952-d12d2c70df05 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:08.706179 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10118ee4-d335-4479-b233-ce9a4345e503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:08.706799 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10118ee4-d335-4479-b233-ce9a4345e503 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:08.707083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10118ee4-d335-4479-b233-ce9a4345e503 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:08.707360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10118ee4-d335-4479-b233-ce9a4345e503 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:08.708060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10118ee4-d335-4479-b233-ce9a4345e503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:08.710309 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10118ee4-d335-4479-b233-ce9a4345e503 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:08.720035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1001/2032] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:57:08.728168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d77fe7-88d7-4bdc-a980-d010407d637b None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:08.728566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d77fe7-88d7-4bdc-a980-d010407d637b None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:08.728842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d77fe7-88d7-4bdc-a980-d010407d637b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:08.748111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-82d77fe7-88d7-4bdc-a980-d010407d637b None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:08.749319 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-82d77fe7-88d7-4bdc-a980-d010407d637b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:08.757236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1002/2033] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:57:08.761259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d77fe7-88d7-4bdc-a980-d010407d637b None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:08.769743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d77fe7-88d7-4bdc-a980-d010407d637b None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:08.774254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6e780-7c98-49f3-bb0a-9d658ff75bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:08.774806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6e780-7c98-49f3-bb0a-9d658ff75bde admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:08.775077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6e780-7c98-49f3-bb0a-9d658ff75bde admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:08.775342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6e780-7c98-49f3-bb0a-9d658ff75bde admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:08.775957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6e780-7c98-49f3-bb0a-9d658ff75bde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:57:08.776256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6e780-7c98-49f3-bb0a-9d658ff75bde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:08.777911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6e780-7c98-49f3-bb0a-9d658ff75bde admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:08.792225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1003/2034] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:57:08.794356 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6e780-7c98-49f3-bb0a-9d658ff75bde admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:08.806766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6e780-7c98-49f3-bb0a-9d658ff75bde admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:08.811375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1032/2035] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:57:08 2022] POST /identity/v3/users => generated 291 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:57:08.812608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42cf7f-8754-439f-b389-499814bf22b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:08.813237 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42cf7f-8754-439f-b389-499814bf22b1 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:08.813514 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42cf7f-8754-439f-b389-499814bf22b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:08.813785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42cf7f-8754-439f-b389-499814bf22b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:08.814474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42cf7f-8754-439f-b389-499814bf22b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:08.815463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10118ee4-d335-4479-b233-ce9a4345e503 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:08.816467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42cf7f-8754-439f-b389-499814bf22b1 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:08.826729 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10118ee4-d335-4479-b233-ce9a4345e503 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:08.832418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cb6b1-1913-48d1-a572-c766fd5fc826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:08.833047 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cb6b1-1913-48d1-a572-c766fd5fc826 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:08.833333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cb6b1-1913-48d1-a572-c766fd5fc826 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:08.833594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1cb6b1-1913-48d1-a572-c766fd5fc826 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:08.834597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cb6b1-1913-48d1-a572-c766fd5fc826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:08.836413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1cb6b1-1913-48d1-a572-c766fd5fc826 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:08.852648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1033/2036] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:08.856642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cb6b1-1913-48d1-a572-c766fd5fc826 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:08.867707 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1cb6b1-1913-48d1-a572-c766fd5fc826 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:08.872752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc4b1d-c6f0-447f-8fba-1389558af5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:08.880193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc4b1d-c6f0-447f-8fba-1389558af5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:08.880193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc4b1d-c6f0-447f-8fba-1389558af5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:08.880193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc4b1d-c6f0-447f-8fba-1389558af5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:08.880193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc4b1d-c6f0-447f-8fba-1389558af5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:08.880193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc4b1d-c6f0-447f-8fba-1389558af5f4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:08.886939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1034/2037] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:57:08.890881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc4b1d-c6f0-447f-8fba-1389558af5f4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:08.900467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc4b1d-c6f0-447f-8fba-1389558af5f4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:08.905078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa28105-2814-406e-a736-97df45eac73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:08.905733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa28105-2814-406e-a736-97df45eac73f admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:08.909074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa28105-2814-406e-a736-97df45eac73f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:08.909074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa28105-2814-406e-a736-97df45eac73f admin admin] PATH_INFO: `/v3/projects/5d8e3f408bde455b86274e11fa626eb4/users/be0920bd9b934f039e5e4de8039346b8/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:08.920977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa28105-2814-406e-a736-97df45eac73f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d8e3f408bde455b86274e11fa626eb4, user_id=be0920bd9b934f039e5e4de8039346b8, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:08.923731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa28105-2814-406e-a736-97df45eac73f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:08.936111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1004/2038] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:57:08 2022] POST /identity/v3/projects => generated 371 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:08.940225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42cf7f-8754-439f-b389-499814bf22b1 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:08.963756 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:08.998237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1035/2039] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:57:08 2022] PUT /identity/v3/projects/5d8e3f408bde455b86274e11fa626eb4/users/be0920bd9b934f039e5e4de8039346b8/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:57:09.001658 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42cf7f-8754-439f-b389-499814bf22b1 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:09.003997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe5185-f4c0-435a-bce9-72541e946958 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:09.004396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe5185-f4c0-435a-bce9-72541e946958 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:09.004672 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe5185-f4c0-435a-bce9-72541e946958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:09.025280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-dcfe5185-f4c0-435a-bce9-72541e946958 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:09.026476 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-dcfe5185-f4c0-435a-bce9-72541e946958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:09.063209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e431b9-7624-4fb1-8303-0a7b63b90c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:09.064017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e431b9-7624-4fb1-8303-0a7b63b90c94 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:09.064403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e431b9-7624-4fb1-8303-0a7b63b90c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:09.064855 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e431b9-7624-4fb1-8303-0a7b63b90c94 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:09.065573 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e431b9-7624-4fb1-8303-0a7b63b90c94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:09.067556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e431b9-7624-4fb1-8303-0a7b63b90c94 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:09.086617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1036/2040] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:57:09.096912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe5185-f4c0-435a-bce9-72541e946958 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:09.098500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-dcfe5185-f4c0-435a-bce9-72541e946958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:09.161254 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe5185-f4c0-435a-bce9-72541e946958 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:09.165337 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d556b3f-931c-4a42-b4c5-0f3cfb894e1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:09.166016 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d556b3f-931c-4a42-b4c5-0f3cfb894e1a service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:09.166294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d556b3f-931c-4a42-b4c5-0f3cfb894e1a service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:09.166579 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d556b3f-931c-4a42-b4c5-0f3cfb894e1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:09.175559 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d556b3f-931c-4a42-b4c5-0f3cfb894e1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:09.177018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d556b3f-931c-4a42-b4c5-0f3cfb894e1a service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:09.186193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1037/2041] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:57:09.218135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1005/2042] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:57:08 2022] POST /identity/v3/users => generated 286 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:57:09.223083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d556b3f-931c-4a42-b4c5-0f3cfb894e1a service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:09.233101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d556b3f-931c-4a42-b4c5-0f3cfb894e1a service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:09.237836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd58cfb-49f1-478c-ae44-f939c0c4a964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:09.238431 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd58cfb-49f1-478c-ae44-f939c0c4a964 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:09.238705 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd58cfb-49f1-478c-ae44-f939c0c4a964 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:09.239065 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd58cfb-49f1-478c-ae44-f939c0c4a964 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:09.239740 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd58cfb-49f1-478c-ae44-f939c0c4a964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:09.241319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd58cfb-49f1-478c-ae44-f939c0c4a964 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:09.252948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1038/2043] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:09.257202 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e431b9-7624-4fb1-8303-0a7b63b90c94 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:09.267682 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e431b9-7624-4fb1-8303-0a7b63b90c94 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:09.273164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974ca241-9f7a-4573-950d-b23c21e8fe86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:09.273776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ca241-9f7a-4573-950d-b23c21e8fe86 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:09.274099 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ca241-9f7a-4573-950d-b23c21e8fe86 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:09.274378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ca241-9f7a-4573-950d-b23c21e8fe86 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:09.275018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ca241-9f7a-4573-950d-b23c21e8fe86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:09.276670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974ca241-9f7a-4573-950d-b23c21e8fe86 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:09.287579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1006/2044] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:09.292315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd58cfb-49f1-478c-ae44-f939c0c4a964 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:09.302682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd58cfb-49f1-478c-ae44-f939c0c4a964 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:09.308553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aaf73d-ba57-473e-b20f-e007cf2924f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:09.310209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaf73d-ba57-473e-b20f-e007cf2924f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:09.310608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaf73d-ba57-473e-b20f-e007cf2924f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:09.311025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaf73d-ba57-473e-b20f-e007cf2924f9 admin admin] PATH_INFO: `/v3/projects/ffbbeaaebb5a4ff39abb3a0173fe6bbc/users/2505792944a14c718d7061fa1281ad50/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:09.313335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:09.330076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aaf73d-ba57-473e-b20f-e007cf2924f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffbbeaaebb5a4ff39abb3a0173fe6bbc, user_id=2505792944a14c718d7061fa1281ad50, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:09.333778 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aaf73d-ba57-473e-b20f-e007cf2924f9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:09.465736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1039/2045] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:57:09 2022] PUT /identity/v3/projects/ffbbeaaebb5a4ff39abb3a0173fe6bbc/users/2505792944a14c718d7061fa1281ad50/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:09.478771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90614e60-b298-4178-ae0d-763cd341168d None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:09.479627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90614e60-b298-4178-ae0d-763cd341168d None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:09.480027 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90614e60-b298-4178-ae0d-763cd341168d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:09.522453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-90614e60-b298-4178-ae0d-763cd341168d None None] MFA Rules not processed for user `2505792944a14c718d7061fa1281ad50`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:09.523861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-90614e60-b298-4178-ae0d-763cd341168d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:09.611158 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1007/2046] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:09.614901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aaf73d-ba57-473e-b20f-e007cf2924f9 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:09.716013 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aaf73d-ba57-473e-b20f-e007cf2924f9 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:09.722591 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34c44d-171d-44a4-8dc6-94e170371983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:09.723724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34c44d-171d-44a4-8dc6-94e170371983 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:09.724250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34c44d-171d-44a4-8dc6-94e170371983 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:09.724643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34c44d-171d-44a4-8dc6-94e170371983 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:09.725429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34c44d-171d-44a4-8dc6-94e170371983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:09.729145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34c44d-171d-44a4-8dc6-94e170371983 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:09.882347 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1040/2047] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:57:09 2022] POST /identity/v3/projects => generated 371 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:09.886647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90614e60-b298-4178-ae0d-763cd341168d None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:09.898390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90614e60-b298-4178-ae0d-763cd341168d None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:09.968790 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b8f62-f2da-47a8-9632-3c9fc40c077d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:09.969704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b8f62-f2da-47a8-9632-3c9fc40c077d admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:09.970075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b8f62-f2da-47a8-9632-3c9fc40c077d admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:09.970348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7b8f62-f2da-47a8-9632-3c9fc40c077d admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:09.971860 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b8f62-f2da-47a8-9632-3c9fc40c077d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:09.974023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7b8f62-f2da-47a8-9632-3c9fc40c077d admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:10.064108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561095e4-e919-4fff-a4ca-221edd5238dd None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:10.064402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561095e4-e919-4fff-a4ca-221edd5238dd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:10.064684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561095e4-e919-4fff-a4ca-221edd5238dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:10.073946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1008/2048] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:57:09 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:57:10.081255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b8f62-f2da-47a8-9632-3c9fc40c077d admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:10.095283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-561095e4-e919-4fff-a4ca-221edd5238dd None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:10.096664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-561095e4-e919-4fff-a4ca-221edd5238dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:10.101728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7b8f62-f2da-47a8-9632-3c9fc40c077d admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:10.107027 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf67fa-9f21-4e83-aa9b-8fa40e2ebd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:10.116846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf67fa-9f21-4e83-aa9b-8fa40e2ebd35 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:10.116846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf67fa-9f21-4e83-aa9b-8fa40e2ebd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:10.116846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabf67fa-9f21-4e83-aa9b-8fa40e2ebd35 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:10.116846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf67fa-9f21-4e83-aa9b-8fa40e2ebd35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:10.116846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabf67fa-9f21-4e83-aa9b-8fa40e2ebd35 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:10.121600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1009/2049] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:10.125765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf67fa-9f21-4e83-aa9b-8fa40e2ebd35 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:10.136314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabf67fa-9f21-4e83-aa9b-8fa40e2ebd35 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:10.145675 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac11fc-890d-40aa-b1f6-369bc0cba2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:10.145675 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac11fc-890d-40aa-b1f6-369bc0cba2a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:10.145675 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac11fc-890d-40aa-b1f6-369bc0cba2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:10.146206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac11fc-890d-40aa-b1f6-369bc0cba2a6 admin admin] PATH_INFO: `/v3/projects/cb2535b9a6d64d9db945f4a022deaefe/users/4103158ad9af4f809e34e83cbd704608/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:10.177371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1041/2050] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 19:57:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:10.183442 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac11fc-890d-40aa-b1f6-369bc0cba2a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb2535b9a6d64d9db945f4a022deaefe, user_id=4103158ad9af4f809e34e83cbd704608, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:10.186694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac11fc-890d-40aa-b1f6-369bc0cba2a6 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:10.193761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561095e4-e919-4fff-a4ca-221edd5238dd None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:10.195215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-561095e4-e919-4fff-a4ca-221edd5238dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:10.267539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561095e4-e919-4fff-a4ca-221edd5238dd None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:10.273950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8135c-7683-491e-a630-18a8efc4f625 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:10.274563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8135c-7683-491e-a630-18a8efc4f625 service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:10.274838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8135c-7683-491e-a630-18a8efc4f625 service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:10.275107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d8135c-7683-491e-a630-18a8efc4f625 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:10.284500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8135c-7683-491e-a630-18a8efc4f625 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:10.286117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d8135c-7683-491e-a630-18a8efc4f625 service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:10.296201 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1042/2051] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 19:57:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 19:57:10.341319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:10.344055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1010/2052] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:57:10 2022] PUT /identity/v3/projects/cb2535b9a6d64d9db945f4a022deaefe/users/4103158ad9af4f809e34e83cbd704608/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:57:10.345730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8135c-7683-491e-a630-18a8efc4f625 service swift] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:10.363113 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d8135c-7683-491e-a630-18a8efc4f625 service swift] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:10.436054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2d8b2-4862-44d0-a26f-7ec1322b84df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:10.436681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2d8b2-4862-44d0-a26f-7ec1322b84df admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:10.436961 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2d8b2-4862-44d0-a26f-7ec1322b84df admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:10.437238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2d8b2-4862-44d0-a26f-7ec1322b84df admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:10.437929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2d8b2-4862-44d0-a26f-7ec1322b84df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:10.439594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2d8b2-4862-44d0-a26f-7ec1322b84df admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:10.450586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1043/2053] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:10.454785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac11fc-890d-40aa-b1f6-369bc0cba2a6 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:10.472474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac11fc-890d-40aa-b1f6-369bc0cba2a6 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:10.477688 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c131a2-eca0-4d9e-b650-b4feaa88998b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:10.478636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c131a2-eca0-4d9e-b650-b4feaa88998b admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:10.478989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c131a2-eca0-4d9e-b650-b4feaa88998b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:10.479329 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c131a2-eca0-4d9e-b650-b4feaa88998b admin admin] PATH_INFO: `/v3/projects/cb2535b9a6d64d9db945f4a022deaefe/users/4103158ad9af4f809e34e83cbd704608/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:10.481315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c131a2-eca0-4d9e-b650-b4feaa88998b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb2535b9a6d64d9db945f4a022deaefe, user_id=4103158ad9af4f809e34e83cbd704608, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:10.484023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c131a2-eca0-4d9e-b650-b4feaa88998b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:10.513153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1011/2054] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:57:10 2022] PUT /identity/v3/projects/cb2535b9a6d64d9db945f4a022deaefe/users/4103158ad9af4f809e34e83cbd704608/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:10.513394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50138153-6bb5-4cb0-93a8-c39c0d8b91f0 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:10.513707 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50138153-6bb5-4cb0-93a8-c39c0d8b91f0 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:10.514009 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50138153-6bb5-4cb0-93a8-c39c0d8b91f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:10.572233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-50138153-6bb5-4cb0-93a8-c39c0d8b91f0 None None] MFA Rules not processed for user `4103158ad9af4f809e34e83cbd704608`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:10.573739 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-50138153-6bb5-4cb0-93a8-c39c0d8b91f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:10.816539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1044/2055] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:10.834113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c131a2-eca0-4d9e-b650-b4feaa88998b admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:10.836277 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a8c131a2-eca0-4d9e-b650-b4feaa88998b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:10.975545 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c131a2-eca0-4d9e-b650-b4feaa88998b admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:10.981271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b4858-2354-48a9-89ca-838dc7e2af7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:10.995565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b4858-2354-48a9-89ca-838dc7e2af7b service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:10.996984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b4858-2354-48a9-89ca-838dc7e2af7b service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:10.998057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b4858-2354-48a9-89ca-838dc7e2af7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:11.032061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b4858-2354-48a9-89ca-838dc7e2af7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:11.033597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b4858-2354-48a9-89ca-838dc7e2af7b service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:11.065399 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1012/2056] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:11.159296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:11.262243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50138153-6bb5-4cb0-93a8-c39c0d8b91f0 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:11.266226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-50138153-6bb5-4cb0-93a8-c39c0d8b91f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:11.419859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50138153-6bb5-4cb0-93a8-c39c0d8b91f0 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:11.425008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61ee92-8787-42ca-b69f-5fd6e59199fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:11.425657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61ee92-8787-42ca-b69f-5fd6e59199fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:11.425967 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61ee92-8787-42ca-b69f-5fd6e59199fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:11.426237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f61ee92-8787-42ca-b69f-5fd6e59199fa admin admin] PATH_INFO: `/v3/users/7bc8055aeedd44b1883652a877200f22` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:11.427422 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61ee92-8787-42ca-b69f-5fd6e59199fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bc8055aeedd44b1883652a877200f22)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:11.429539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f61ee92-8787-42ca-b69f-5fd6e59199fa admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:11.805859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f61ee92-8787-42ca-b69f-5fd6e59199fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bc8055aeedd44b1883652a877200f22'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:11.938335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f61ee92-8787-42ca-b69f-5fd6e59199fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bc8055aeedd44b1883652a877200f22'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:11.973056 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f61ee92-8787-42ca-b69f-5fd6e59199fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bc8055aeedd44b1883652a877200f22'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:11.974490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f61ee92-8787-42ca-b69f-5fd6e59199fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bc8055aeedd44b1883652a877200f22'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:11.976715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f61ee92-8787-42ca-b69f-5fd6e59199fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bc8055aeedd44b1883652a877200f22'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:12.022050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1045/2057] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:57:11 2022] DELETE /identity/v3/users/7bc8055aeedd44b1883652a877200f22 => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:12.028005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b3c67-e6ed-420d-a539-2979c6f2b233 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:12.028526 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b3c67-e6ed-420d-a539-2979c6f2b233 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:12.028607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b3c67-e6ed-420d-a539-2979c6f2b233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:12.102696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-8b9b3c67-e6ed-420d-a539-2979c6f2b233 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:12.105723 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8b9b3c67-e6ed-420d-a539-2979c6f2b233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:12.301406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:12.367085 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1013/2058] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:12.399798 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61ee92-8787-42ca-b69f-5fd6e59199fa admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:12.401638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6f61ee92-8787-42ca-b69f-5fd6e59199fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:12.549070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f61ee92-8787-42ca-b69f-5fd6e59199fa admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:12.572965 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950277f-8cac-44aa-bd7e-c1e4599ef6b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:12.573881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950277f-8cac-44aa-bd7e-c1e4599ef6b0 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:12.574275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950277f-8cac-44aa-bd7e-c1e4599ef6b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:12.574728 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950277f-8cac-44aa-bd7e-c1e4599ef6b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:12.606230 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950277f-8cac-44aa-bd7e-c1e4599ef6b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:12.613418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950277f-8cac-44aa-bd7e-c1e4599ef6b0 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:12.634179 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1046/2059] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:12.792733 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b3c67-e6ed-420d-a539-2979c6f2b233 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:12.794649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8b9b3c67-e6ed-420d-a539-2979c6f2b233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:12.882642 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b3c67-e6ed-420d-a539-2979c6f2b233 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:12.892269 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c1ad2-cc95-427f-920c-dbcaa35e0779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:12.893128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1ad2-cc95-427f-920c-dbcaa35e0779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:12.893466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1ad2-cc95-427f-920c-dbcaa35e0779 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:12.894038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1ad2-cc95-427f-920c-dbcaa35e0779 admin admin] PATH_INFO: `/v3/users/2505792944a14c718d7061fa1281ad50` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:12.895761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c1ad2-cc95-427f-920c-dbcaa35e0779 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2505792944a14c718d7061fa1281ad50)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:12.897867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c1ad2-cc95-427f-920c-dbcaa35e0779 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:13.240668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c54c1ad2-cc95-427f-920c-dbcaa35e0779 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2505792944a14c718d7061fa1281ad50'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:13.282043 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c54c1ad2-cc95-427f-920c-dbcaa35e0779 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2505792944a14c718d7061fa1281ad50'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:13.352999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:13.386642 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c54c1ad2-cc95-427f-920c-dbcaa35e0779 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2505792944a14c718d7061fa1281ad50'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:13.387717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c54c1ad2-cc95-427f-920c-dbcaa35e0779 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2505792944a14c718d7061fa1281ad50'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:13.404091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c54c1ad2-cc95-427f-920c-dbcaa35e0779 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2505792944a14c718d7061fa1281ad50'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:13.426604 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1014/2060] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:57:12 2022] DELETE /identity/v3/users/2505792944a14c718d7061fa1281ad50 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:13.431683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b14f83-d0a1-4a4b-aab1-0f6459b1b8de None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:13.432189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b14f83-d0a1-4a4b-aab1-0f6459b1b8de None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:13.432267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b14f83-d0a1-4a4b-aab1-0f6459b1b8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:13.478258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b1b14f83-d0a1-4a4b-aab1-0f6459b1b8de None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:13.479992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b1b14f83-d0a1-4a4b-aab1-0f6459b1b8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:13.747878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1047/2061] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:13.750304 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c1ad2-cc95-427f-920c-dbcaa35e0779 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:13.753734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c54c1ad2-cc95-427f-920c-dbcaa35e0779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:13.871973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c1ad2-cc95-427f-920c-dbcaa35e0779 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:13.871973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8850c-22c0-497d-a5b3-168bc01af5ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:13.871973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8850c-22c0-497d-a5b3-168bc01af5ad service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:13.871973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8850c-22c0-497d-a5b3-168bc01af5ad service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:13.875364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8850c-22c0-497d-a5b3-168bc01af5ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:13.881170 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8850c-22c0-497d-a5b3-168bc01af5ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:13.883253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8850c-22c0-497d-a5b3-168bc01af5ad service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:13.905534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1015/2062] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:14.488514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b14f83-d0a1-4a4b-aab1-0f6459b1b8de None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:14.490603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b1b14f83-d0a1-4a4b-aab1-0f6459b1b8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:14.522391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b14f83-d0a1-4a4b-aab1-0f6459b1b8de None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:14.536359 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:14.537393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:14.537935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:14.538511 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] PATH_INFO: `/v3/projects/756419ab2bc64870a61c623ba2316e1d` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:14.539879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] RBAC: Authorizing `identity:delete_project(project_id=756419ab2bc64870a61c623ba2316e1d)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:14.544107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:14.773050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '756419ab2bc64870a61c623ba2316e1d', 'request_id': 'req-73a89b28-5b43-4f72-aadb-c395ff9c040c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:14.906676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '756419ab2bc64870a61c623ba2316e1d', 'request_id': 'req-73a89b28-5b43-4f72-aadb-c395ff9c040c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:14.919559 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '756419ab2bc64870a61c623ba2316e1d', 'request_id': 'req-73a89b28-5b43-4f72-aadb-c395ff9c040c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:14.940395 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] The token cache is being invalidate because project 756419ab2bc64870a61c623ba2316e1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:57:14.940752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:14.941486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:14.943418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1048/2063] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:57:14 2022] DELETE /identity/v3/projects/756419ab2bc64870a61c623ba2316e1d => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:14.947524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8850c-22c0-497d-a5b3-168bc01af5ad service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:14.960345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8850c-22c0-497d-a5b3-168bc01af5ad service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:15.077070 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9cf4a-e9c6-4ee3-a4da-0512833228b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:15.077834 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9cf4a-e9c6-4ee3-a4da-0512833228b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:15.078225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9cf4a-e9c6-4ee3-a4da-0512833228b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:15.079234 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9cf4a-e9c6-4ee3-a4da-0512833228b7 admin admin] PATH_INFO: `/v3/users/be78381048f84b34b99ebc2bc6961b23` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:15.079994 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9cf4a-e9c6-4ee3-a4da-0512833228b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be78381048f84b34b99ebc2bc6961b23)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:15.082844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9cf4a-e9c6-4ee3-a4da-0512833228b7 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:15.151929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:15.154495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:15.172961 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a89b28-5b43-4f72-aadb-c395ff9c040c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:15.188500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:15.189382 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:15.189900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:15.190342 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] PATH_INFO: `/v3/projects/ffbbeaaebb5a4ff39abb3a0173fe6bbc` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:15.192915 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffbbeaaebb5a4ff39abb3a0173fe6bbc)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:15.195047 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:15.250985 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:15.384927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-daa9cf4a-e9c6-4ee3-a4da-0512833228b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be78381048f84b34b99ebc2bc6961b23'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:15.415384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-daa9cf4a-e9c6-4ee3-a4da-0512833228b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be78381048f84b34b99ebc2bc6961b23'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:15.431652 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffbbeaaebb5a4ff39abb3a0173fe6bbc', 'request_id': 'req-56629683-d5a3-4748-9916-f91ee0d23c5b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:15.490218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-daa9cf4a-e9c6-4ee3-a4da-0512833228b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be78381048f84b34b99ebc2bc6961b23'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:15.490982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-daa9cf4a-e9c6-4ee3-a4da-0512833228b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be78381048f84b34b99ebc2bc6961b23'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:15.509525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-daa9cf4a-e9c6-4ee3-a4da-0512833228b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be78381048f84b34b99ebc2bc6961b23'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:15.527537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1016/2064] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:57:14 2022] DELETE /identity/v3/users/be78381048f84b34b99ebc2bc6961b23 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:15.586230 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffbbeaaebb5a4ff39abb3a0173fe6bbc', 'request_id': 'req-56629683-d5a3-4748-9916-f91ee0d23c5b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:15.594388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffbbeaaebb5a4ff39abb3a0173fe6bbc', 'request_id': 'req-56629683-d5a3-4748-9916-f91ee0d23c5b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:15.624796 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] The token cache is being invalidate because project ffbbeaaebb5a4ff39abb3a0173fe6bbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:57:15.624796 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:15.624796 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:15.628191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1049/2065] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:57:15 2022] DELETE /identity/v3/projects/ffbbeaaebb5a4ff39abb3a0173fe6bbc => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:15.631082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9cf4a-e9c6-4ee3-a4da-0512833228b7 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:15.633295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-daa9cf4a-e9c6-4ee3-a4da-0512833228b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:15.771002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9cf4a-e9c6-4ee3-a4da-0512833228b7 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:15.777633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1b1069-50dc-4091-8d8f-89a9ece80b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:15.778395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b1069-50dc-4091-8d8f-89a9ece80b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:15.778772 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b1069-50dc-4091-8d8f-89a9ece80b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:15.779120 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1b1069-50dc-4091-8d8f-89a9ece80b6f admin admin] PATH_INFO: `/v3/users/4103158ad9af4f809e34e83cbd704608` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:15.780433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b1069-50dc-4091-8d8f-89a9ece80b6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4103158ad9af4f809e34e83cbd704608)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:15.783058 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1b1069-50dc-4091-8d8f-89a9ece80b6f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:15.995338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4f1b1069-50dc-4091-8d8f-89a9ece80b6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4103158ad9af4f809e34e83cbd704608'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:16.004207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:16.025250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4f1b1069-50dc-4091-8d8f-89a9ece80b6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4103158ad9af4f809e34e83cbd704608'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:16.123261 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4f1b1069-50dc-4091-8d8f-89a9ece80b6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4103158ad9af4f809e34e83cbd704608'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:16.125020 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4f1b1069-50dc-4091-8d8f-89a9ece80b6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4103158ad9af4f809e34e83cbd704608'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:16.138349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4f1b1069-50dc-4091-8d8f-89a9ece80b6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4103158ad9af4f809e34e83cbd704608'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:16.154849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1017/2066] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:57:15 2022] DELETE /identity/v3/users/4103158ad9af4f809e34e83cbd704608 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:57:16.187447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:16.188752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:16.334901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56629683-d5a3-4748-9916-f91ee0d23c5b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:16.466142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5791854-4058-4bb9-a9b9-3019a60ed14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:16.466886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5791854-4058-4bb9-a9b9-3019a60ed14d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:16.467165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5791854-4058-4bb9-a9b9-3019a60ed14d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:16.467447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5791854-4058-4bb9-a9b9-3019a60ed14d admin admin] PATH_INFO: `/v3/projects/008134341efb416faeffabc5c3dda2c3` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:16.470669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5791854-4058-4bb9-a9b9-3019a60ed14d admin admin] RBAC: Authorizing `identity:delete_project(project_id=008134341efb416faeffabc5c3dda2c3)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:16.489623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5791854-4058-4bb9-a9b9-3019a60ed14d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:16.674337 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e5791854-4058-4bb9-a9b9-3019a60ed14d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '008134341efb416faeffabc5c3dda2c3', 'request_id': 'req-e5791854-4058-4bb9-a9b9-3019a60ed14d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:16.778026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e5791854-4058-4bb9-a9b9-3019a60ed14d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '008134341efb416faeffabc5c3dda2c3', 'request_id': 'req-e5791854-4058-4bb9-a9b9-3019a60ed14d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:16.792384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e5791854-4058-4bb9-a9b9-3019a60ed14d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '008134341efb416faeffabc5c3dda2c3', 'request_id': 'req-e5791854-4058-4bb9-a9b9-3019a60ed14d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:16.819071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e5791854-4058-4bb9-a9b9-3019a60ed14d admin admin] The token cache is being invalidate because project 008134341efb416faeffabc5c3dda2c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:57:16.819425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e5791854-4058-4bb9-a9b9-3019a60ed14d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:16.820549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e5791854-4058-4bb9-a9b9-3019a60ed14d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:16.822359 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1050/2067] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:57:16 2022] DELETE /identity/v3/projects/008134341efb416faeffabc5c3dda2c3 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:16.828039 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db5d03-b251-4260-9ecf-756197510f4a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:16.828325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db5d03-b251-4260-9ecf-756197510f4a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:16.828592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85db5d03-b251-4260-9ecf-756197510f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:16.890025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-85db5d03-b251-4260-9ecf-756197510f4a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:16.891292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-85db5d03-b251-4260-9ecf-756197510f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:16.981521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:16.993097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1018/2068] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:17.001543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6813d-8fd6-43f8-a89d-7ef5d6e6aba5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:17.001870 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6813d-8fd6-43f8-a89d-7ef5d6e6aba5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:17.002171 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6813d-8fd6-43f8-a89d-7ef5d6e6aba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:17.060705 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-57a6813d-8fd6-43f8-a89d-7ef5d6e6aba5 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:17.062439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-57a6813d-8fd6-43f8-a89d-7ef5d6e6aba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:17.075246 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1051/2069] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:17.079102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db5d03-b251-4260-9ecf-756197510f4a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:17.094456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85db5d03-b251-4260-9ecf-756197510f4a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:17.105911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d3608-ccf6-43a3-8e6f-dbf7c6561d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:17.115642 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d3608-ccf6-43a3-8e6f-dbf7c6561d44 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:17.118084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d3608-ccf6-43a3-8e6f-dbf7c6561d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:17.118084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921d3608-ccf6-43a3-8e6f-dbf7c6561d44 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:17.125965 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d3608-ccf6-43a3-8e6f-dbf7c6561d44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:57:17.128705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d3608-ccf6-43a3-8e6f-dbf7c6561d44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:17.130707 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921d3608-ccf6-43a3-8e6f-dbf7c6561d44 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:17.166452 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6813d-8fd6-43f8-a89d-7ef5d6e6aba5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:17.174785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1019/2070] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:57:17.202376 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6813d-8fd6-43f8-a89d-7ef5d6e6aba5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:17.227265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90749f-579a-48ad-9f65-15bae05a13f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:17.227265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90749f-579a-48ad-9f65-15bae05a13f4 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:17.227615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90749f-579a-48ad-9f65-15bae05a13f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:17.227999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90749f-579a-48ad-9f65-15bae05a13f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:17.228721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90749f-579a-48ad-9f65-15bae05a13f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:17.230925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90749f-579a-48ad-9f65-15bae05a13f4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:17.358263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d3608-ccf6-43a3-8e6f-dbf7c6561d44 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:17.361465 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:17.369079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-921d3608-ccf6-43a3-8e6f-dbf7c6561d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:17.378272 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1052/2071] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:57:17 2022] POST /identity/v3/projects => generated 373 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:17.390413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90749f-579a-48ad-9f65-15bae05a13f4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:17.400739 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921d3608-ccf6-43a3-8e6f-dbf7c6561d44 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:17.540926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fd37c-43df-4140-b128-5443bce5c38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:17.542496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fd37c-43df-4140-b128-5443bce5c38c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:17.542983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fd37c-43df-4140-b128-5443bce5c38c admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:17.543452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fd37c-43df-4140-b128-5443bce5c38c admin admin] PATH_INFO: `/v3/projects/cb2535b9a6d64d9db945f4a022deaefe` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:17.545073 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fd37c-43df-4140-b128-5443bce5c38c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb2535b9a6d64d9db945f4a022deaefe)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:17.547756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fd37c-43df-4140-b128-5443bce5c38c admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:17.601934 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90749f-579a-48ad-9f65-15bae05a13f4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:17.609151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bca69-204e-46ec-8860-4332022f926f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:17.610217 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bca69-204e-46ec-8860-4332022f926f admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:17.610678 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bca69-204e-46ec-8860-4332022f926f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:17.610951 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9bca69-204e-46ec-8860-4332022f926f admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:17.611638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bca69-204e-46ec-8860-4332022f926f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:17.615199 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9bca69-204e-46ec-8860-4332022f926f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:17.748656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d74fd37c-43df-4140-b128-5443bce5c38c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb2535b9a6d64d9db945f4a022deaefe', 'request_id': 'req-d74fd37c-43df-4140-b128-5443bce5c38c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:17.769039 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d74fd37c-43df-4140-b128-5443bce5c38c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb2535b9a6d64d9db945f4a022deaefe', 'request_id': 'req-d74fd37c-43df-4140-b128-5443bce5c38c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:17.790847 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1053/2072] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:57:17 2022] POST /identity/v3/users => generated 287 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:17.794859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bca69-204e-46ec-8860-4332022f926f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:17.797929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d74fd37c-43df-4140-b128-5443bce5c38c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb2535b9a6d64d9db945f4a022deaefe', 'request_id': 'req-d74fd37c-43df-4140-b128-5443bce5c38c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:17.818564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9bca69-204e-46ec-8860-4332022f926f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:17.912517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d74fd37c-43df-4140-b128-5443bce5c38c admin admin] The token cache is being invalidate because project cb2535b9a6d64d9db945f4a022deaefe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:57:17.912812 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d74fd37c-43df-4140-b128-5443bce5c38c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:17.913712 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d74fd37c-43df-4140-b128-5443bce5c38c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:17.915424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1020/2073] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:57:17 2022] DELETE /identity/v3/projects/cb2535b9a6d64d9db945f4a022deaefe => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:57:17.923322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c25ae-0824-4a6a-b4dd-a6cc711ff634 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:17.923608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c25ae-0824-4a6a-b4dd-a6cc711ff634 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:17.923876 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c25ae-0824-4a6a-b4dd-a6cc711ff634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:17.926479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548a8bd-b568-436b-97cc-6bee954e019c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:17.927102 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548a8bd-b568-436b-97cc-6bee954e019c admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:17.927435 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548a8bd-b568-436b-97cc-6bee954e019c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:17.927708 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548a8bd-b568-436b-97cc-6bee954e019c admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:17.928403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548a8bd-b568-436b-97cc-6bee954e019c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:17.930912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548a8bd-b568-436b-97cc-6bee954e019c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:17.945479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1054/2074] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:17.950241 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548a8bd-b568-436b-97cc-6bee954e019c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:17.952401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b548a8bd-b568-436b-97cc-6bee954e019c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:17.974573 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-517c25ae-0824-4a6a-b4dd-a6cc711ff634 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:17.976033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-517c25ae-0824-4a6a-b4dd-a6cc711ff634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:17.977495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548a8bd-b568-436b-97cc-6bee954e019c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:17.982984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efe660-502d-455d-a8a4-bce79d6aef4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:17.983994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe660-502d-455d-a8a4-bce79d6aef4e admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:17.984392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe660-502d-455d-a8a4-bce79d6aef4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:17.984822 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efe660-502d-455d-a8a4-bce79d6aef4e admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:17.985629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efe660-502d-455d-a8a4-bce79d6aef4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:17.987820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efe660-502d-455d-a8a4-bce79d6aef4e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:18.000903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1021/2075] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:18.009251 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb89ae-c8d5-452e-95fe-b6217966d8cf None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:18.012161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb89ae-c8d5-452e-95fe-b6217966d8cf None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:18.012339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1055/2076] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:57:18.012467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb89ae-c8d5-452e-95fe-b6217966d8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:18.019686 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efe660-502d-455d-a8a4-bce79d6aef4e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:18.041035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-86cb89ae-c8d5-452e-95fe-b6217966d8cf None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:18.042482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-86cb89ae-c8d5-452e-95fe-b6217966d8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:18.043827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efe660-502d-455d-a8a4-bce79d6aef4e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:18.056500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9d680-9ab8-476f-b3ff-c2566bd09469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:18.058516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9d680-9ab8-476f-b3ff-c2566bd09469 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:18.058796 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9d680-9ab8-476f-b3ff-c2566bd09469 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:18.059224 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9d680-9ab8-476f-b3ff-c2566bd09469 admin admin] PATH_INFO: `/v3/projects/5517c245f9da455c94de364623252018/users/defc5e3f630e4076bb0086b1314975fe/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:18.077562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1022/2077] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:57:18.078376 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9d680-9ab8-476f-b3ff-c2566bd09469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5517c245f9da455c94de364623252018, user_id=defc5e3f630e4076bb0086b1314975fe, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:18.081684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9d680-9ab8-476f-b3ff-c2566bd09469 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:18.082290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb89ae-c8d5-452e-95fe-b6217966d8cf None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:18.091668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb89ae-c8d5-452e-95fe-b6217966d8cf None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:18.098387 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565480e-2de6-4a83-8441-0d7a87d31c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:18.098988 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565480e-2de6-4a83-8441-0d7a87d31c13 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:18.099360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565480e-2de6-4a83-8441-0d7a87d31c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:18.099578 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3565480e-2de6-4a83-8441-0d7a87d31c13 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:18.100215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565480e-2de6-4a83-8441-0d7a87d31c13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:57:18.100561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565480e-2de6-4a83-8441-0d7a87d31c13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:18.102170 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3565480e-2de6-4a83-8441-0d7a87d31c13 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:18.114425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1023/2078] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:57:18.126536 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565480e-2de6-4a83-8441-0d7a87d31c13 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:18.136529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3565480e-2de6-4a83-8441-0d7a87d31c13 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:18.141579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a22a2-1992-40df-9668-58b21ca8372f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:18.142217 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a22a2-1992-40df-9668-58b21ca8372f admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:18.142500 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a22a2-1992-40df-9668-58b21ca8372f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:18.142778 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a22a2-1992-40df-9668-58b21ca8372f admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:18.143944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a22a2-1992-40df-9668-58b21ca8372f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:18.145866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375a22a2-1992-40df-9668-58b21ca8372f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:18.251854 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1056/2079] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:57:18 2022] PUT /identity/v3/projects/5517c245f9da455c94de364623252018/users/defc5e3f630e4076bb0086b1314975fe/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:18.266387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654b8a1-82dc-4d18-a507-426c2b066f6c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:18.267084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654b8a1-82dc-4d18-a507-426c2b066f6c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:18.268786 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7654b8a1-82dc-4d18-a507-426c2b066f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:18.294097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1024/2080] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:57:18 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:57:18.298001 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a22a2-1992-40df-9668-58b21ca8372f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:18.315647 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7654b8a1-82dc-4d18-a507-426c2b066f6c None None] MFA Rules not processed for user `defc5e3f630e4076bb0086b1314975fe`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:18.317303 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7654b8a1-82dc-4d18-a507-426c2b066f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:18.393500 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a22a2-1992-40df-9668-58b21ca8372f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:18.397625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1057/2081] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:18.407672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c7ebf-11c3-444e-bac2-b4fbc667af68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:18.408367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c7ebf-11c3-444e-bac2-b4fbc667af68 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:18.409499 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c7ebf-11c3-444e-bac2-b4fbc667af68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:18.409793 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c7ebf-11c3-444e-bac2-b4fbc667af68 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:18.410234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654b8a1-82dc-4d18-a507-426c2b066f6c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:18.410435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c7ebf-11c3-444e-bac2-b4fbc667af68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:18.411766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7654b8a1-82dc-4d18-a507-426c2b066f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:18.412649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c7ebf-11c3-444e-bac2-b4fbc667af68 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:18.484344 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7654b8a1-82dc-4d18-a507-426c2b066f6c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:18.496799 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15834996-6d67-4df1-804d-f79d80134307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:18.498215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15834996-6d67-4df1-804d-f79d80134307 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:18.498711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15834996-6d67-4df1-804d-f79d80134307 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:18.499122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15834996-6d67-4df1-804d-f79d80134307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:18.511018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15834996-6d67-4df1-804d-f79d80134307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:18.512814 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15834996-6d67-4df1-804d-f79d80134307 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:18.525100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1058/2082] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:57:18.526574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1025/2083] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:57:18 2022] POST /identity/v3/users => generated 291 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:18.531167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15834996-6d67-4df1-804d-f79d80134307 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:18.564445 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c7ebf-11c3-444e-bac2-b4fbc667af68 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:18.576697 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c7ebf-11c3-444e-bac2-b4fbc667af68 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:18.577121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15834996-6d67-4df1-804d-f79d80134307 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:18.582692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6875cdd5-4906-4733-b33d-47450c294558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:18.583512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875cdd5-4906-4733-b33d-47450c294558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:18.583941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875cdd5-4906-4733-b33d-47450c294558 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:18.584530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6875cdd5-4906-4733-b33d-47450c294558 admin admin] PATH_INFO: `/v3/users/defc5e3f630e4076bb0086b1314975fe` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:18.586072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875cdd5-4906-4733-b33d-47450c294558 admin admin] RBAC: Authorizing `identity:delete_user(user_id=defc5e3f630e4076bb0086b1314975fe)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:18.588409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281abb23-9e4f-444e-b70b-c3f342557585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:18.588588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6875cdd5-4906-4733-b33d-47450c294558 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:18.589051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281abb23-9e4f-444e-b70b-c3f342557585 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:18.589333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281abb23-9e4f-444e-b70b-c3f342557585 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:18.590098 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281abb23-9e4f-444e-b70b-c3f342557585 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:18.590813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281abb23-9e4f-444e-b70b-c3f342557585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:18.592749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281abb23-9e4f-444e-b70b-c3f342557585 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:18.610966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1059/2084] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:57:18.614797 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281abb23-9e4f-444e-b70b-c3f342557585 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:18.629080 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281abb23-9e4f-444e-b70b-c3f342557585 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:18.634393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cffcbf-48b5-4e76-96d9-aab03fdd88f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:18.635986 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cffcbf-48b5-4e76-96d9-aab03fdd88f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:18.636426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cffcbf-48b5-4e76-96d9-aab03fdd88f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:18.637393 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cffcbf-48b5-4e76-96d9-aab03fdd88f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:18.638699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cffcbf-48b5-4e76-96d9-aab03fdd88f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:18.641152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cffcbf-48b5-4e76-96d9-aab03fdd88f5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:18.660820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1060/2085] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:18.662307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cffcbf-48b5-4e76-96d9-aab03fdd88f5 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:18.682473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cffcbf-48b5-4e76-96d9-aab03fdd88f5 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:18.693594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa292c2-a9ef-4b17-94cb-2a81b8afcd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:18.694259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa292c2-a9ef-4b17-94cb-2a81b8afcd74 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:18.694535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa292c2-a9ef-4b17-94cb-2a81b8afcd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:18.694809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa292c2-a9ef-4b17-94cb-2a81b8afcd74 admin admin] PATH_INFO: `/v3/projects/318ee0cb8ee7481abe3e94cc7f9c94b0/users/94fb9135499a441583db2565d133be0b/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:18.724839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:18.732907 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa292c2-a9ef-4b17-94cb-2a81b8afcd74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=318ee0cb8ee7481abe3e94cc7f9c94b0, user_id=94fb9135499a441583db2565d133be0b, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:18.747042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa292c2-a9ef-4b17-94cb-2a81b8afcd74 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:18.877267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1061/2086] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:57:18 2022] PUT /identity/v3/projects/318ee0cb8ee7481abe3e94cc7f9c94b0/users/94fb9135499a441583db2565d133be0b/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:57:18.907520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6875cdd5-4906-4733-b33d-47450c294558 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'defc5e3f630e4076bb0086b1314975fe'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:18.911210 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f9ad8-3a1e-473f-891d-6d210ccab8bd None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:18.912033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f9ad8-3a1e-473f-891d-6d210ccab8bd None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:18.912033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f9ad8-3a1e-473f-891d-6d210ccab8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:18.939513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6875cdd5-4906-4733-b33d-47450c294558 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'defc5e3f630e4076bb0086b1314975fe'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:18.975107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-444f9ad8-3a1e-473f-891d-6d210ccab8bd None None] MFA Rules not processed for user `94fb9135499a441583db2565d133be0b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:18.976384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-444f9ad8-3a1e-473f-891d-6d210ccab8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:19.026958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6875cdd5-4906-4733-b33d-47450c294558 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'defc5e3f630e4076bb0086b1314975fe'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:19.027891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6875cdd5-4906-4733-b33d-47450c294558 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'defc5e3f630e4076bb0086b1314975fe'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:19.050128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6875cdd5-4906-4733-b33d-47450c294558 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'defc5e3f630e4076bb0086b1314975fe'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:19.072583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1026/2087] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:57:18 2022] DELETE /identity/v3/users/defc5e3f630e4076bb0086b1314975fe => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:19.076743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0bc5d-82c6-4cb9-a16e-68977cb5d183 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:19.077117 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0bc5d-82c6-4cb9-a16e-68977cb5d183 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:19.078227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0bc5d-82c6-4cb9-a16e-68977cb5d183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:19.141893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-d2c0bc5d-82c6-4cb9-a16e-68977cb5d183 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:19.144274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d2c0bc5d-82c6-4cb9-a16e-68977cb5d183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:19.165034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1062/2088] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:19.170142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f9ad8-3a1e-473f-891d-6d210ccab8bd None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:19.173076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-444f9ad8-3a1e-473f-891d-6d210ccab8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:19.278993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1027/2089] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:19.283580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc253494-4ab1-4624-b417-5e210628b795 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:19.284784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc253494-4ab1-4624-b417-5e210628b795 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:19.285067 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc253494-4ab1-4624-b417-5e210628b795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:19.330133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-bc253494-4ab1-4624-b417-5e210628b795 None None] MFA Rules not processed for user `be0920bd9b934f039e5e4de8039346b8`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:19.331977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bc253494-4ab1-4624-b417-5e210628b795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:19.342367 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f9ad8-3a1e-473f-891d-6d210ccab8bd None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:19.363021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2790c-7cbf-40ae-876b-b4a02bbfd483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:19.363742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2790c-7cbf-40ae-876b-b4a02bbfd483 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:19.365768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2790c-7cbf-40ae-876b-b4a02bbfd483 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:19.366129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2790c-7cbf-40ae-876b-b4a02bbfd483 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:19.366785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2790c-7cbf-40ae-876b-b4a02bbfd483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:19.370554 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe2790c-7cbf-40ae-876b-b4a02bbfd483 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:19.448266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1028/2090] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:19.449369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc253494-4ab1-4624-b417-5e210628b795 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:19.453141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bc253494-4ab1-4624-b417-5e210628b795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:19.483672 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1063/2091] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:57:19 2022] POST /identity/v3/projects => generated 383 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:19.484566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2790c-7cbf-40ae-876b-b4a02bbfd483 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:19.489278 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-dbe2790c-7cbf-40ae-876b-b4a02bbfd483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:19.586402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc253494-4ab1-4624-b417-5e210628b795 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:19.644820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe2790c-7cbf-40ae-876b-b4a02bbfd483 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:19.667719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea9e08-ee17-4ad1-ab91-6e220afd864e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:19.671175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea9e08-ee17-4ad1-ab91-6e220afd864e service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:19.671175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea9e08-ee17-4ad1-ab91-6e220afd864e service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:19.671175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bea9e08-ee17-4ad1-ab91-6e220afd864e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:19.700577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50d51c-f623-4b68-8ff4-37f7c810c7d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:19.714878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50d51c-f623-4b68-8ff4-37f7c810c7d6 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:19.714878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50d51c-f623-4b68-8ff4-37f7c810c7d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:19.714878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b50d51c-f623-4b68-8ff4-37f7c810c7d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:19.790752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea9e08-ee17-4ad1-ab91-6e220afd864e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:19.794998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bea9e08-ee17-4ad1-ab91-6e220afd864e service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:19.813286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1064/2092] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:19.814397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea9e08-ee17-4ad1-ab91-6e220afd864e service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:19.823019 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50d51c-f623-4b68-8ff4-37f7c810c7d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:19.824478 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b50d51c-f623-4b68-8ff4-37f7c810c7d6 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:19.828764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bea9e08-ee17-4ad1-ab91-6e220afd864e service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:19.836007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8be4e-9600-4205-8aed-507d174ddb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:19.836984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8be4e-9600-4205-8aed-507d174ddb79 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:19.837197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1029/2093] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:19.838165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8be4e-9600-4205-8aed-507d174ddb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:19.838745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f8be4e-9600-4205-8aed-507d174ddb79 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:19.841297 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8be4e-9600-4205-8aed-507d174ddb79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:19.852459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f8be4e-9600-4205-8aed-507d174ddb79 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:20.036349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1065/2094] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:57:19 2022] POST /identity/v3/users => generated 298 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:20.040089 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50d51c-f623-4b68-8ff4-37f7c810c7d6 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:20.053124 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b50d51c-f623-4b68-8ff4-37f7c810c7d6 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:20.066186 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b99a1f-05a6-4f75-be05-0e610f1117b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:20.066863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b99a1f-05a6-4f75-be05-0e610f1117b2 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:20.067132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b99a1f-05a6-4f75-be05-0e610f1117b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:20.067417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b99a1f-05a6-4f75-be05-0e610f1117b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:20.068054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b99a1f-05a6-4f75-be05-0e610f1117b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:20.069663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b99a1f-05a6-4f75-be05-0e610f1117b2 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:20.105840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1030/2095] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:20.107128 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8be4e-9600-4205-8aed-507d174ddb79 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:20.120062 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f8be4e-9600-4205-8aed-507d174ddb79 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:20.126487 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312df46-00d7-472d-afc5-064cb96709ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:20.127339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312df46-00d7-472d-afc5-064cb96709ee admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:20.128615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312df46-00d7-472d-afc5-064cb96709ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:20.129049 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312df46-00d7-472d-afc5-064cb96709ee admin admin] PATH_INFO: `/v3/projects/ce02b929d0394091a6cc5da56e42e521/users/7d5a19e67bbe42ec9fd38dc56efd7f27/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:20.204455 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312df46-00d7-472d-afc5-064cb96709ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce02b929d0394091a6cc5da56e42e521, user_id=7d5a19e67bbe42ec9fd38dc56efd7f27, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:20.220273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312df46-00d7-472d-afc5-064cb96709ee admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:20.339618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1066/2096] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:57:20 2022] PUT /identity/v3/projects/ce02b929d0394091a6cc5da56e42e521/users/7d5a19e67bbe42ec9fd38dc56efd7f27/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:20.344051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b99a1f-05a6-4f75-be05-0e610f1117b2 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:20.360487 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b99a1f-05a6-4f75-be05-0e610f1117b2 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:20.472480 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a162d62-f2f2-4541-bfc7-1bb06abf8ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:20.486436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a162d62-f2f2-4541-bfc7-1bb06abf8ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:20.487077 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a162d62-f2f2-4541-bfc7-1bb06abf8ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:20.487459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a162d62-f2f2-4541-bfc7-1bb06abf8ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:20.488300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a162d62-f2f2-4541-bfc7-1bb06abf8ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:20.490512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a162d62-f2f2-4541-bfc7-1bb06abf8ae3 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:20.512661 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:20.520682 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1031/2097] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:20.525280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312df46-00d7-472d-afc5-064cb96709ee admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:20.539021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312df46-00d7-472d-afc5-064cb96709ee admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:20.548222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7b1ee-1ff8-434a-abc9-7a8d1b626811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:20.548222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7b1ee-1ff8-434a-abc9-7a8d1b626811 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:20.548222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7b1ee-1ff8-434a-abc9-7a8d1b626811 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:20.548222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7b1ee-1ff8-434a-abc9-7a8d1b626811 admin admin] PATH_INFO: `/v3/projects/ce02b929d0394091a6cc5da56e42e521/users/7d5a19e67bbe42ec9fd38dc56efd7f27/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:20.551311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7b1ee-1ff8-434a-abc9-7a8d1b626811 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce02b929d0394091a6cc5da56e42e521, user_id=7d5a19e67bbe42ec9fd38dc56efd7f27, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:20.554615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7b1ee-1ff8-434a-abc9-7a8d1b626811 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:20.570785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1067/2098] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:57:20 2022] PUT /identity/v3/projects/ce02b929d0394091a6cc5da56e42e521/users/7d5a19e67bbe42ec9fd38dc56efd7f27/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:20.584474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3e4a7-2952-4b47-837a-3918d8681c95 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:20.584840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3e4a7-2952-4b47-837a-3918d8681c95 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:20.585320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f3e4a7-2952-4b47-837a-3918d8681c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:20.667424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e0f3e4a7-2952-4b47-837a-3918d8681c95 None None] MFA Rules not processed for user `7d5a19e67bbe42ec9fd38dc56efd7f27`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:20.669456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e0f3e4a7-2952-4b47-837a-3918d8681c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:20.738650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7b1ee-1ff8-434a-abc9-7a8d1b626811 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:20.740924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c1a7b1ee-1ff8-434a-abc9-7a8d1b626811 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:20.931137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1032/2099] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:20.953784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3e4a7-2952-4b47-837a-3918d8681c95 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:20.953784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e0f3e4a7-2952-4b47-837a-3918d8681c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:20.998381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7b1ee-1ff8-434a-abc9-7a8d1b626811 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:21.017839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3137ff2-c921-4b78-9053-f6f2315f10f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:21.018531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3137ff2-c921-4b78-9053-f6f2315f10f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:21.018814 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3137ff2-c921-4b78-9053-f6f2315f10f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:21.019089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3137ff2-c921-4b78-9053-f6f2315f10f4 admin admin] PATH_INFO: `/v3/projects/5517c245f9da455c94de364623252018` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:21.023149 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3137ff2-c921-4b78-9053-f6f2315f10f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5517c245f9da455c94de364623252018)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:21.025187 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3137ff2-c921-4b78-9053-f6f2315f10f4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:21.178360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f3e4a7-2952-4b47-837a-3918d8681c95 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:21.188252 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5bc37-6b2b-498f-9937-97a6326d5692 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:21.189095 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5bc37-6b2b-498f-9937-97a6326d5692 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:21.189379 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5bc37-6b2b-498f-9937-97a6326d5692 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:21.189653 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5bc37-6b2b-498f-9937-97a6326d5692 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:21.215266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5bc37-6b2b-498f-9937-97a6326d5692 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:21.216746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5bc37-6b2b-498f-9937-97a6326d5692 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:21.236572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1033/2100] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:20 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:21.304371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b3137ff2-c921-4b78-9053-f6f2315f10f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5517c245f9da455c94de364623252018', 'request_id': 'req-b3137ff2-c921-4b78-9053-f6f2315f10f4'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:21.429498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b3137ff2-c921-4b78-9053-f6f2315f10f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5517c245f9da455c94de364623252018', 'request_id': 'req-b3137ff2-c921-4b78-9053-f6f2315f10f4'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:21.454419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5bc37-6b2b-498f-9937-97a6326d5692 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:21.478043 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5bc37-6b2b-498f-9937-97a6326d5692 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:21.478248 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b3137ff2-c921-4b78-9053-f6f2315f10f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5517c245f9da455c94de364623252018', 'request_id': 'req-b3137ff2-c921-4b78-9053-f6f2315f10f4'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:21.526731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b3137ff2-c921-4b78-9053-f6f2315f10f4 admin admin] The token cache is being invalidate because project 5517c245f9da455c94de364623252018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:57:21.527114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b3137ff2-c921-4b78-9053-f6f2315f10f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:21.529626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b3137ff2-c921-4b78-9053-f6f2315f10f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:21.532601 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1068/2101] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:57:20 2022] DELETE /identity/v3/projects/5517c245f9da455c94de364623252018 => generated 0 bytes in 795 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:21.538567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1dfd2-fe39-40d2-9be1-ef4f9cf6b200 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:21.538857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1dfd2-fe39-40d2-9be1-ef4f9cf6b200 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:21.539165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1dfd2-fe39-40d2-9be1-ef4f9cf6b200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:21.610203 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31f01c-6597-476f-9fe3-4c3de4e93a2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:21.610319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-04a1dfd2-fe39-40d2-9be1-ef4f9cf6b200 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:21.610430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31f01c-6597-476f-9fe3-4c3de4e93a2e service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:21.610430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31f01c-6597-476f-9fe3-4c3de4e93a2e service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:21.610430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31f01c-6597-476f-9fe3-4c3de4e93a2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:21.610628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-04a1dfd2-fe39-40d2-9be1-ef4f9cf6b200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:21.632944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-7c31f01c-6597-476f-9fe3-4c3de4e93a2e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:21.718226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:21.792263 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1069/2102] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:21.800890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7473e5-7fea-4ffe-a3f5-5a3f7be495be None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:21.801177 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7473e5-7fea-4ffe-a3f5-5a3f7be495be None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:21.801448 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7473e5-7fea-4ffe-a3f5-5a3f7be495be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:21.817143 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31f01c-6597-476f-9fe3-4c3de4e93a2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:21.818680 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c31f01c-6597-476f-9fe3-4c3de4e93a2e service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:21.827386 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-cd7473e5-7fea-4ffe-a3f5-5a3f7be495be None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:21.828914 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cd7473e5-7fea-4ffe-a3f5-5a3f7be495be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:21.843553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1070/2103] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:21.852695 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7473e5-7fea-4ffe-a3f5-5a3f7be495be None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:21.855756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1034/2104] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:21.881101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7473e5-7fea-4ffe-a3f5-5a3f7be495be None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:21.904685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ba805-57c8-4032-802e-930b7a131cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:21.905532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ba805-57c8-4032-802e-930b7a131cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:21.906334 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ba805-57c8-4032-802e-930b7a131cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:21.906921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ba805-57c8-4032-802e-930b7a131cb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:21.907925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ba805-57c8-4032-802e-930b7a131cb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:57:21.908408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ba805-57c8-4032-802e-930b7a131cb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:21.911348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ba805-57c8-4032-802e-930b7a131cb3 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:21.929842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1071/2105] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:57:21.942722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31f01c-6597-476f-9fe3-4c3de4e93a2e service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:21.944605 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31f01c-6597-476f-9fe3-4c3de4e93a2e service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:21.949939 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e291e6-f719-4551-9247-a25720f69726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:21.950630 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e291e6-f719-4551-9247-a25720f69726 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:21.950973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e291e6-f719-4551-9247-a25720f69726 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:21.951311 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e291e6-f719-4551-9247-a25720f69726 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:21.952083 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e291e6-f719-4551-9247-a25720f69726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:21.954196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e291e6-f719-4551-9247-a25720f69726 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:22.105237 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1035/2106] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:57:21 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:57:22.110096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ba805-57c8-4032-802e-930b7a131cb3 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:22.125312 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ba805-57c8-4032-802e-930b7a131cb3 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:22.230168 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781bb7a-9d05-4dd3-a447-773249c24d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:22.231037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781bb7a-9d05-4dd3-a447-773249c24d5d admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:22.231458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781bb7a-9d05-4dd3-a447-773249c24d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:22.232400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6781bb7a-9d05-4dd3-a447-773249c24d5d admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:22.233397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781bb7a-9d05-4dd3-a447-773249c24d5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:22.235905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6781bb7a-9d05-4dd3-a447-773249c24d5d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:22.353598 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1072/2107] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:57:22 2022] POST /identity/v3/users => generated 292 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:22.357618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e291e6-f719-4551-9247-a25720f69726 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:22.371243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e291e6-f719-4551-9247-a25720f69726 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:22.377477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30558f3b-ef67-47af-963c-64f50532744f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:22.378400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30558f3b-ef67-47af-963c-64f50532744f admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:22.378758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30558f3b-ef67-47af-963c-64f50532744f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:22.379114 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30558f3b-ef67-47af-963c-64f50532744f admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:22.379901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30558f3b-ef67-47af-963c-64f50532744f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:22.382613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30558f3b-ef67-47af-963c-64f50532744f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:22.397917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1036/2108] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:22.402577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781bb7a-9d05-4dd3-a447-773249c24d5d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:22.414653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6781bb7a-9d05-4dd3-a447-773249c24d5d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:22.429971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e50fa2-9455-4869-9207-a7d7c2a82999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:22.430598 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e50fa2-9455-4869-9207-a7d7c2a82999 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:22.430876 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e50fa2-9455-4869-9207-a7d7c2a82999 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:22.431140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e50fa2-9455-4869-9207-a7d7c2a82999 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:22.431775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e50fa2-9455-4869-9207-a7d7c2a82999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:22.433439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e50fa2-9455-4869-9207-a7d7c2a82999 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:22.444549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1073/2109] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 19:57:22.449083 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30558f3b-ef67-47af-963c-64f50532744f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:22.459112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30558f3b-ef67-47af-963c-64f50532744f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:22.464179 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f3ad8-7437-4ff1-9bf4-dc274bb0c43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:22.464873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f3ad8-7437-4ff1-9bf4-dc274bb0c43a admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:22.465151 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f3ad8-7437-4ff1-9bf4-dc274bb0c43a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:22.465424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f3ad8-7437-4ff1-9bf4-dc274bb0c43a admin admin] PATH_INFO: `/v3/projects/ea79258eb56c40fea9bf19e4c8fff327/users/4fc50ddf89014d3c92851af9b62eb15c/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:22.488460 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f3ad8-7437-4ff1-9bf4-dc274bb0c43a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea79258eb56c40fea9bf19e4c8fff327, user_id=4fc50ddf89014d3c92851af9b62eb15c, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:22.492306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f3ad8-7437-4ff1-9bf4-dc274bb0c43a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:22.670325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1037/2110] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:57:22 2022] PUT /identity/v3/projects/ea79258eb56c40fea9bf19e4c8fff327/users/4fc50ddf89014d3c92851af9b62eb15c/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:22.687030 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f06a7e-646c-4e9e-bd07-380505b5c4b8 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:22.687352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f06a7e-646c-4e9e-bd07-380505b5c4b8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:22.687628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f06a7e-646c-4e9e-bd07-380505b5c4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:22.730464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f3ad8-7437-4ff1-9bf4-dc274bb0c43a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:22.732642 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-745f3ad8-7437-4ff1-9bf4-dc274bb0c43a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:22.745681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-87f06a7e-646c-4e9e-bd07-380505b5c4b8 None None] MFA Rules not processed for user `4fc50ddf89014d3c92851af9b62eb15c`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:22.750381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-87f06a7e-646c-4e9e-bd07-380505b5c4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:22.860796 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1074/2111] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:22.878150 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f06a7e-646c-4e9e-bd07-380505b5c4b8 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:22.878640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-87f06a7e-646c-4e9e-bd07-380505b5c4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:22.895339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f3ad8-7437-4ff1-9bf4-dc274bb0c43a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:22.911615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7aa10-5043-42f2-86b9-200e7abcd4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:22.911981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7aa10-5043-42f2-86b9-200e7abcd4c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:22.912416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7aa10-5043-42f2-86b9-200e7abcd4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:22.912780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7aa10-5043-42f2-86b9-200e7abcd4c2 admin admin] PATH_INFO: `/v3/users/94fb9135499a441583db2565d133be0b` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:22.917032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7aa10-5043-42f2-86b9-200e7abcd4c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94fb9135499a441583db2565d133be0b)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:22.919393 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7aa10-5043-42f2-86b9-200e7abcd4c2 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:22.982247 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f06a7e-646c-4e9e-bd07-380505b5c4b8 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:22.987144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28969978-cd92-4f3a-8163-67e1c423d278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:22.987776 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28969978-cd92-4f3a-8163-67e1c423d278 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:22.988062 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28969978-cd92-4f3a-8163-67e1c423d278 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:22.988361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28969978-cd92-4f3a-8163-67e1c423d278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:23.000552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28969978-cd92-4f3a-8163-67e1c423d278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:23.002997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28969978-cd92-4f3a-8163-67e1c423d278 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:23.015710 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1075/2112] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:23.242502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0eb7aa10-5043-42f2-86b9-200e7abcd4c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94fb9135499a441583db2565d133be0b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:23.259719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0eb7aa10-5043-42f2-86b9-200e7abcd4c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94fb9135499a441583db2565d133be0b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:23.346886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0eb7aa10-5043-42f2-86b9-200e7abcd4c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94fb9135499a441583db2565d133be0b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:23.347536 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0eb7aa10-5043-42f2-86b9-200e7abcd4c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94fb9135499a441583db2565d133be0b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:23.356898 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0eb7aa10-5043-42f2-86b9-200e7abcd4c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94fb9135499a441583db2565d133be0b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:23.367207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1038/2113] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:57:22 2022] DELETE /identity/v3/users/94fb9135499a441583db2565d133be0b => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:23.370924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd688ad-4712-4db2-aace-3f31829addf8 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:23.371301 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd688ad-4712-4db2-aace-3f31829addf8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:23.371495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd688ad-4712-4db2-aace-3f31829addf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:23.397320 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1bd688ad-4712-4db2-aace-3f31829addf8 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:23.398801 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1bd688ad-4712-4db2-aace-3f31829addf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:23.495791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1076/2114] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:23.509837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7aa10-5043-42f2-86b9-200e7abcd4c2 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:23.514413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0eb7aa10-5043-42f2-86b9-200e7abcd4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:23.594138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7aa10-5043-42f2-86b9-200e7abcd4c2 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:23.598733 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cdae0d-185b-4c78-b314-2fc445a7ea25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:23.599338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cdae0d-185b-4c78-b314-2fc445a7ea25 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:23.599611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cdae0d-185b-4c78-b314-2fc445a7ea25 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:23.599872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cdae0d-185b-4c78-b314-2fc445a7ea25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:23.603263 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe2fc4-c9cc-4d33-b7f6-99cf58d5afe6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:23.603573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe2fc4-c9cc-4d33-b7f6-99cf58d5afe6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:23.603852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe2fc4-c9cc-4d33-b7f6-99cf58d5afe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:23.616034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cdae0d-185b-4c78-b314-2fc445a7ea25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:23.619241 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cdae0d-185b-4c78-b314-2fc445a7ea25 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:23.635238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-96fe2fc4-c9cc-4d33-b7f6-99cf58d5afe6 None None] MFA Rules not processed for user `be0920bd9b934f039e5e4de8039346b8`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:23.636620 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-96fe2fc4-c9cc-4d33-b7f6-99cf58d5afe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:23.641202 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1039/2115] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:23.797996 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1077/2116] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:57:23.862902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:24.301929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cdae0d-185b-4c78-b314-2fc445a7ea25 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:24.303520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-08cdae0d-185b-4c78-b314-2fc445a7ea25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:24.348524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cdae0d-185b-4c78-b314-2fc445a7ea25 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:24.368827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:24.369573 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:24.379820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:24.380413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] PATH_INFO: `/v3/projects/318ee0cb8ee7481abe3e94cc7f9c94b0` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:24.385841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=318ee0cb8ee7481abe3e94cc7f9c94b0)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:24.392133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:24.583721 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '318ee0cb8ee7481abe3e94cc7f9c94b0', 'request_id': 'req-07ca71ee-db5b-4cab-a37e-344635ec67e7'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:24.602367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '318ee0cb8ee7481abe3e94cc7f9c94b0', 'request_id': 'req-07ca71ee-db5b-4cab-a37e-344635ec67e7'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:24.612041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '318ee0cb8ee7481abe3e94cc7f9c94b0', 'request_id': 'req-07ca71ee-db5b-4cab-a37e-344635ec67e7'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:24.707132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] The token cache is being invalidate because project 318ee0cb8ee7481abe3e94cc7f9c94b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:57:24.707544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:24.710968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:24.713954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1040/2117] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:57:24 2022] DELETE /identity/v3/projects/318ee0cb8ee7481abe3e94cc7f9c94b0 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:24.720593 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe2fc4-c9cc-4d33-b7f6-99cf58d5afe6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:24.743382 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe2fc4-c9cc-4d33-b7f6-99cf58d5afe6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:24.885435 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f1234-0478-4d08-9b4c-03cc057e33f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:24.888959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f1234-0478-4d08-9b4c-03cc057e33f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:24.888959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f1234-0478-4d08-9b4c-03cc057e33f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:24.888959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f1234-0478-4d08-9b4c-03cc057e33f0 admin admin] PATH_INFO: `/v3/users/7d5a19e67bbe42ec9fd38dc56efd7f27` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:24.890144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f1234-0478-4d08-9b4c-03cc057e33f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d5a19e67bbe42ec9fd38dc56efd7f27)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:24.898091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f1234-0478-4d08-9b4c-03cc057e33f0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:25.042594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:25.157924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b05f1234-0478-4d08-9b4c-03cc057e33f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d5a19e67bbe42ec9fd38dc56efd7f27'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:25.289332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b05f1234-0478-4d08-9b4c-03cc057e33f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d5a19e67bbe42ec9fd38dc56efd7f27'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:25.303154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b05f1234-0478-4d08-9b4c-03cc057e33f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d5a19e67bbe42ec9fd38dc56efd7f27'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:25.303893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b05f1234-0478-4d08-9b4c-03cc057e33f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d5a19e67bbe42ec9fd38dc56efd7f27'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:25.304857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b05f1234-0478-4d08-9b4c-03cc057e33f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d5a19e67bbe42ec9fd38dc56efd7f27'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:25.319857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1078/2118] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:57:24 2022] DELETE /identity/v3/users/7d5a19e67bbe42ec9fd38dc56efd7f27 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:25.827516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:25.830445 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:25.854574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca71ee-db5b-4cab-a37e-344635ec67e7 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:25.872809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:25.937454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238dcdc9-1f67-4a37-bdab-3302b1b90af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:25.938128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dcdc9-1f67-4a37-bdab-3302b1b90af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:25.938408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dcdc9-1f67-4a37-bdab-3302b1b90af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:25.938685 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238dcdc9-1f67-4a37-bdab-3302b1b90af2 admin admin] PATH_INFO: `/v3/projects/ce02b929d0394091a6cc5da56e42e521` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:25.940900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238dcdc9-1f67-4a37-bdab-3302b1b90af2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce02b929d0394091a6cc5da56e42e521)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:25.942883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238dcdc9-1f67-4a37-bdab-3302b1b90af2 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:26.101968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-238dcdc9-1f67-4a37-bdab-3302b1b90af2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce02b929d0394091a6cc5da56e42e521', 'request_id': 'req-238dcdc9-1f67-4a37-bdab-3302b1b90af2'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:26.107628 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-238dcdc9-1f67-4a37-bdab-3302b1b90af2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce02b929d0394091a6cc5da56e42e521', 'request_id': 'req-238dcdc9-1f67-4a37-bdab-3302b1b90af2'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:26.114697 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-238dcdc9-1f67-4a37-bdab-3302b1b90af2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce02b929d0394091a6cc5da56e42e521', 'request_id': 'req-238dcdc9-1f67-4a37-bdab-3302b1b90af2'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:26.225893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-238dcdc9-1f67-4a37-bdab-3302b1b90af2 admin admin] The token cache is being invalidate because project ce02b929d0394091a6cc5da56e42e521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:57:26.226184 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-238dcdc9-1f67-4a37-bdab-3302b1b90af2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:26.226826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-238dcdc9-1f67-4a37-bdab-3302b1b90af2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:26.228578 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1041/2119] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:57:25 2022] DELETE /identity/v3/projects/ce02b929d0394091a6cc5da56e42e521 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:26.233753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b54296-14b4-419b-9d9a-ff61b658e78e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:26.234358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b54296-14b4-419b-9d9a-ff61b658e78e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:26.234449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b54296-14b4-419b-9d9a-ff61b658e78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:26.254562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-30b54296-14b4-419b-9d9a-ff61b658e78e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:26.255687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-30b54296-14b4-419b-9d9a-ff61b658e78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:26.321866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1079/2120] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:26.330544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c0419-9ee1-4bfc-a9fe-85626165602b None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:26.330825 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c0419-9ee1-4bfc-a9fe-85626165602b None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:26.331102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8c0419-9ee1-4bfc-a9fe-85626165602b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:26.351062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-1e8c0419-9ee1-4bfc-a9fe-85626165602b None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:26.355226 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1e8c0419-9ee1-4bfc-a9fe-85626165602b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:26.364141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1042/2121] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:26.368607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b54296-14b4-419b-9d9a-ff61b658e78e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:26.378838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b54296-14b4-419b-9d9a-ff61b658e78e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:26.383560 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1a2df-209e-46ba-bbdb-20b306dad866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:26.384153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1a2df-209e-46ba-bbdb-20b306dad866 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:26.384459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1a2df-209e-46ba-bbdb-20b306dad866 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:26.384735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee1a2df-209e-46ba-bbdb-20b306dad866 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:26.385370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1a2df-209e-46ba-bbdb-20b306dad866 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:57:26.385669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1a2df-209e-46ba-bbdb-20b306dad866 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:26.387292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee1a2df-209e-46ba-bbdb-20b306dad866 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:26.399050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1080/2122] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:57:26.404023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c0419-9ee1-4bfc-a9fe-85626165602b None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:26.413282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8c0419-9ee1-4bfc-a9fe-85626165602b None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:26.417904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9b788-991b-492b-94ee-85d57ad6dcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:26.418488 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9b788-991b-492b-94ee-85d57ad6dcd1 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:26.418762 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9b788-991b-492b-94ee-85d57ad6dcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:26.419027 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9b788-991b-492b-94ee-85d57ad6dcd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:26.419765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9b788-991b-492b-94ee-85d57ad6dcd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:26.421659 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9b788-991b-492b-94ee-85d57ad6dcd1 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:26.591408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1043/2123] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:57:26 2022] POST /identity/v3/projects => generated 375 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:57:26.595632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1a2df-209e-46ba-bbdb-20b306dad866 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:26.606008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee1a2df-209e-46ba-bbdb-20b306dad866 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:26.660469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d6698-1f03-4554-8108-b340b20daa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:26.661082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d6698-1f03-4554-8108-b340b20daa39 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:26.661336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d6698-1f03-4554-8108-b340b20daa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:26.661602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d6698-1f03-4554-8108-b340b20daa39 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:26.662278 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d6698-1f03-4554-8108-b340b20daa39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:26.665024 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d6698-1f03-4554-8108-b340b20daa39 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:26.790507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1081/2124] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:57:26 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:26.794543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9b788-991b-492b-94ee-85d57ad6dcd1 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:26.805208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9b788-991b-492b-94ee-85d57ad6dcd1 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:26.809824 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf385748-cba2-4e49-a9f5-b36a5cd503ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:26.810434 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf385748-cba2-4e49-a9f5-b36a5cd503ba admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:26.810690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf385748-cba2-4e49-a9f5-b36a5cd503ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:26.811045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf385748-cba2-4e49-a9f5-b36a5cd503ba admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:26.811745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf385748-cba2-4e49-a9f5-b36a5cd503ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:26.813384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf385748-cba2-4e49-a9f5-b36a5cd503ba admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:26.823162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1044/2125] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:26.827109 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d6698-1f03-4554-8108-b340b20daa39 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:26.836875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d6698-1f03-4554-8108-b340b20daa39 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:26.841731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942c39f-09a3-43e8-b699-80a328a183b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:26.842438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942c39f-09a3-43e8-b699-80a328a183b5 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:26.842711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942c39f-09a3-43e8-b699-80a328a183b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:26.842978 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942c39f-09a3-43e8-b699-80a328a183b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:26.843610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942c39f-09a3-43e8-b699-80a328a183b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:26.845229 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942c39f-09a3-43e8-b699-80a328a183b5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:26.856636 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1082/2126] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:26.860775 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf385748-cba2-4e49-a9f5-b36a5cd503ba admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:26.872139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf385748-cba2-4e49-a9f5-b36a5cd503ba admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:26.877654 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa8bd7-068d-4347-a90e-b3e559a53bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:26.878346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa8bd7-068d-4347-a90e-b3e559a53bd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:26.878622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa8bd7-068d-4347-a90e-b3e559a53bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:26.878890 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa8bd7-068d-4347-a90e-b3e559a53bd9 admin admin] PATH_INFO: `/v3/projects/049f7df0aaba4dd784680e73e5d70283/users/e762f35d4e3345949a37fca67d10da36/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:26.893629 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa8bd7-068d-4347-a90e-b3e559a53bd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=049f7df0aaba4dd784680e73e5d70283, user_id=e762f35d4e3345949a37fca67d10da36, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:26.896344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa8bd7-068d-4347-a90e-b3e559a53bd9 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:27.074528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1045/2127] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:57:26 2022] PUT /identity/v3/projects/049f7df0aaba4dd784680e73e5d70283/users/e762f35d4e3345949a37fca67d10da36/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:57:27.085085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749da50-ed07-46f5-94c1-a9d6fa2c7b6a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:27.085903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749da50-ed07-46f5-94c1-a9d6fa2c7b6a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:27.086324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1749da50-ed07-46f5-94c1-a9d6fa2c7b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:27.109273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:27.120022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1749da50-ed07-46f5-94c1-a9d6fa2c7b6a None None] MFA Rules not processed for user `e762f35d4e3345949a37fca67d10da36`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:27.121388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1749da50-ed07-46f5-94c1-a9d6fa2c7b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:27.223449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1083/2128] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:27.230940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa8bd7-068d-4347-a90e-b3e559a53bd9 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:27.232704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-dafa8bd7-068d-4347-a90e-b3e559a53bd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:27.294522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa8bd7-068d-4347-a90e-b3e559a53bd9 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:27.298138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1f940-3aa4-4509-987c-8d14694fc68d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:27.298733 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1f940-3aa4-4509-987c-8d14694fc68d service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:27.299006 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1f940-3aa4-4509-987c-8d14694fc68d service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:27.299275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1f940-3aa4-4509-987c-8d14694fc68d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:27.307671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1f940-3aa4-4509-987c-8d14694fc68d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:27.309156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1f940-3aa4-4509-987c-8d14694fc68d service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:27.318427 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1046/2129] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:27.901465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:28.404598 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:28.964414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:29.344384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:30.327565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:30.499015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:30.552947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:30.770327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:30.824140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=4,keystone:SELECT=90,keystone:INSERT=6 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:31.024754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:31.481629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:31.942623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:32.095608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:34.256398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:34.407908 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:35.055927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:35.082501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:35.267247 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:35.307051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:35.661754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:35.753328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:35.782051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:35.938979 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:35.973237 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:36.381893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:36.413391 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:37.433042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749da50-ed07-46f5-94c1-a9d6fa2c7b6a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:37.495361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1749da50-ed07-46f5-94c1-a9d6fa2c7b6a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:37.499950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b8996-e5aa-4562-ab62-898efa2d89ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:37.500661 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b8996-e5aa-4562-ab62-898efa2d89ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:37.500944 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b8996-e5aa-4562-ab62-898efa2d89ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:37.501211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b8996-e5aa-4562-ab62-898efa2d89ee admin admin] PATH_INFO: `/v3/users/e762f35d4e3345949a37fca67d10da36` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:37.502369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b8996-e5aa-4562-ab62-898efa2d89ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=e762f35d4e3345949a37fca67d10da36)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:37.504305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b8996-e5aa-4562-ab62-898efa2d89ee admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:37.625428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:37.707584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:37.759629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ad6b8996-e5aa-4562-ab62-898efa2d89ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e762f35d4e3345949a37fca67d10da36'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:37.874310 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ad6b8996-e5aa-4562-ab62-898efa2d89ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e762f35d4e3345949a37fca67d10da36'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:37.890150 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ad6b8996-e5aa-4562-ab62-898efa2d89ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e762f35d4e3345949a37fca67d10da36'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:37.890919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ad6b8996-e5aa-4562-ab62-898efa2d89ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e762f35d4e3345949a37fca67d10da36'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:37.891526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ad6b8996-e5aa-4562-ab62-898efa2d89ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e762f35d4e3345949a37fca67d10da36'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:37.911314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1084/2130] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:57:37 2022] DELETE /identity/v3/users/e762f35d4e3345949a37fca67d10da36 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:37.913267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effd7a2-f769-438a-adf8-1859effc1d90 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:37.913560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effd7a2-f769-438a-adf8-1859effc1d90 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:37.913995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effd7a2-f769-438a-adf8-1859effc1d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:37.939671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-1effd7a2-f769-438a-adf8-1859effc1d90 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:37.942227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1effd7a2-f769-438a-adf8-1859effc1d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:38.011729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1047/2131] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:38.023950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b8996-e5aa-4562-ab62-898efa2d89ee admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:38.025496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ad6b8996-e5aa-4562-ab62-898efa2d89ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:38.056285 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:38.089995 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b8996-e5aa-4562-ab62-898efa2d89ee admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:38.093594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6caf4-2ae9-4597-8330-d41c2999ff04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:38.094258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6caf4-2ae9-4597-8330-d41c2999ff04 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:38.094592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6caf4-2ae9-4597-8330-d41c2999ff04 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:38.094871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc6caf4-2ae9-4597-8330-d41c2999ff04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:38.104940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6caf4-2ae9-4597-8330-d41c2999ff04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:38.106436 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc6caf4-2ae9-4597-8330-d41c2999ff04 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:38.116926 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1085/2132] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:38.704289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effd7a2-f769-438a-adf8-1859effc1d90 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:38.705852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1effd7a2-f769-438a-adf8-1859effc1d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:38.720293 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effd7a2-f769-438a-adf8-1859effc1d90 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:38.726343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff954a-60a8-4339-96a6-4955fa5e967b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:38.727051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff954a-60a8-4339-96a6-4955fa5e967b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:38.727447 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff954a-60a8-4339-96a6-4955fa5e967b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:38.727832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff954a-60a8-4339-96a6-4955fa5e967b admin admin] PATH_INFO: `/v3/users/4fc50ddf89014d3c92851af9b62eb15c` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:38.731188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff954a-60a8-4339-96a6-4955fa5e967b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fc50ddf89014d3c92851af9b62eb15c)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:38.733531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff954a-60a8-4339-96a6-4955fa5e967b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:38.890439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6caf4-2ae9-4597-8330-d41c2999ff04 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:38.894093 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5bc6caf4-2ae9-4597-8330-d41c2999ff04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:38.912313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc6caf4-2ae9-4597-8330-d41c2999ff04 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:38.920426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fad6c89-79ca-4b07-b2a9-6225db633745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:38.920426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad6c89-79ca-4b07-b2a9-6225db633745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:38.920426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad6c89-79ca-4b07-b2a9-6225db633745 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:38.920426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fad6c89-79ca-4b07-b2a9-6225db633745 admin admin] PATH_INFO: `/v3/projects/049f7df0aaba4dd784680e73e5d70283` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:38.920747 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad6c89-79ca-4b07-b2a9-6225db633745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=049f7df0aaba4dd784680e73e5d70283)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:38.921997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fad6c89-79ca-4b07-b2a9-6225db633745 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:38.990564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c8ff954a-60a8-4339-96a6-4955fa5e967b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fc50ddf89014d3c92851af9b62eb15c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:39.004913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c8ff954a-60a8-4339-96a6-4955fa5e967b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fc50ddf89014d3c92851af9b62eb15c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:39.043840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5fad6c89-79ca-4b07-b2a9-6225db633745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '049f7df0aaba4dd784680e73e5d70283', 'request_id': 'req-5fad6c89-79ca-4b07-b2a9-6225db633745'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:39.110259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c8ff954a-60a8-4339-96a6-4955fa5e967b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fc50ddf89014d3c92851af9b62eb15c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:39.110950 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c8ff954a-60a8-4339-96a6-4955fa5e967b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fc50ddf89014d3c92851af9b62eb15c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:39.121329 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c8ff954a-60a8-4339-96a6-4955fa5e967b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fc50ddf89014d3c92851af9b62eb15c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:39.129676 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1048/2133] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:57:38 2022] DELETE /identity/v3/users/4fc50ddf89014d3c92851af9b62eb15c => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:39.134743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844051e-095c-474b-91bd-65eae3fb6d44 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:39.135033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844051e-095c-474b-91bd-65eae3fb6d44 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:39.135313 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c844051e-095c-474b-91bd-65eae3fb6d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:39.144414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:39.163466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-c844051e-095c-474b-91bd-65eae3fb6d44 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:39.164893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c844051e-095c-474b-91bd-65eae3fb6d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:39.204474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5fad6c89-79ca-4b07-b2a9-6225db633745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '049f7df0aaba4dd784680e73e5d70283', 'request_id': 'req-5fad6c89-79ca-4b07-b2a9-6225db633745'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:39.215165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5fad6c89-79ca-4b07-b2a9-6225db633745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '049f7df0aaba4dd784680e73e5d70283', 'request_id': 'req-5fad6c89-79ca-4b07-b2a9-6225db633745'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:39.233664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5fad6c89-79ca-4b07-b2a9-6225db633745 admin admin] The token cache is being invalidate because project 049f7df0aaba4dd784680e73e5d70283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:57:39.234303 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5fad6c89-79ca-4b07-b2a9-6225db633745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:39.242419 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5fad6c89-79ca-4b07-b2a9-6225db633745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:39.245356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1086/2134] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:57:38 2022] DELETE /identity/v3/projects/049f7df0aaba4dd784680e73e5d70283 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:39.251083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1c0ad-edd1-4257-97f9-4bfcb9ad92f8 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:39.253028 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1c0ad-edd1-4257-97f9-4bfcb9ad92f8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:39.255477 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1c0ad-edd1-4257-97f9-4bfcb9ad92f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:39.260728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1049/2135] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:39.271687 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844051e-095c-474b-91bd-65eae3fb6d44 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:39.275014 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c844051e-095c-474b-91bd-65eae3fb6d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:39.276103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6ef1c0ad-edd1-4257-97f9-4bfcb9ad92f8 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:39.277472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6ef1c0ad-edd1-4257-97f9-4bfcb9ad92f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:39.292921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1087/2136] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:39.300626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559f44d-42d1-453f-af02-055b550a05ee None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:39.300908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559f44d-42d1-453f-af02-055b550a05ee None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:39.301185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559f44d-42d1-453f-af02-055b550a05ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:39.320526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7559f44d-42d1-453f-af02-055b550a05ee None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:39.321724 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7559f44d-42d1-453f-af02-055b550a05ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:39.330295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1088/2137] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:57:39.334798 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559f44d-42d1-453f-af02-055b550a05ee None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:39.342237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:39.345079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c844051e-095c-474b-91bd-65eae3fb6d44 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:39.345916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7559f44d-42d1-453f-af02-055b550a05ee None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:39.349371 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52852ded-0b96-462a-90e7-ec1e18a9039e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:39.351236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52852ded-0b96-462a-90e7-ec1e18a9039e service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:39.351236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52852ded-0b96-462a-90e7-ec1e18a9039e service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:39.351236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52852ded-0b96-462a-90e7-ec1e18a9039e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:39.351496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881e169-2fe9-466f-b3aa-73dbc2b35757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:39.351703 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881e169-2fe9-466f-b3aa-73dbc2b35757 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:39.352063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881e169-2fe9-466f-b3aa-73dbc2b35757 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:39.352337 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d881e169-2fe9-466f-b3aa-73dbc2b35757 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:39.352973 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881e169-2fe9-466f-b3aa-73dbc2b35757 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:57:39.353274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881e169-2fe9-466f-b3aa-73dbc2b35757 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:39.354933 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d881e169-2fe9-466f-b3aa-73dbc2b35757 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:39.359736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52852ded-0b96-462a-90e7-ec1e18a9039e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:39.361136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52852ded-0b96-462a-90e7-ec1e18a9039e service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:39.366700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1089/2138] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 19:57:39.371291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881e169-2fe9-466f-b3aa-73dbc2b35757 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:39.371807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1050/2139] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:57:39.391569 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d881e169-2fe9-466f-b3aa-73dbc2b35757 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:39.399589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3191c-6fe5-4d88-8395-fa73b9053722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:39.400190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3191c-6fe5-4d88-8395-fa73b9053722 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:39.400469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3191c-6fe5-4d88-8395-fa73b9053722 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:39.400742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3191c-6fe5-4d88-8395-fa73b9053722 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:39.401377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3191c-6fe5-4d88-8395-fa73b9053722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:39.404155 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a3191c-6fe5-4d88-8395-fa73b9053722 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:39.573523 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1090/2140] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:57:39 2022] POST /identity/v3/projects => generated 365 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:39.574753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52852ded-0b96-462a-90e7-ec1e18a9039e service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:39.587641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52852ded-0b96-462a-90e7-ec1e18a9039e service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:39.642304 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c116d-0fa4-42fc-9180-62d18300e778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:39.642931 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c116d-0fa4-42fc-9180-62d18300e778 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:39.643214 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c116d-0fa4-42fc-9180-62d18300e778 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:39.644137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c116d-0fa4-42fc-9180-62d18300e778 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:39.644778 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c116d-0fa4-42fc-9180-62d18300e778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:39.646766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c116d-0fa4-42fc-9180-62d18300e778 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:39.681891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:39.764869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1051/2141] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:57:39 2022] POST /identity/v3/users => generated 283 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:39.766094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3191c-6fe5-4d88-8395-fa73b9053722 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:39.776190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3191c-6fe5-4d88-8395-fa73b9053722 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:39.780812 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eca4a-6d8a-4c6f-921c-97ac51cf1ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:39.781401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eca4a-6d8a-4c6f-921c-97ac51cf1ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:39.781674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eca4a-6d8a-4c6f-921c-97ac51cf1ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:39.781987 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1eca4a-6d8a-4c6f-921c-97ac51cf1ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:39.782620 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1eca4a-6d8a-4c6f-921c-97ac51cf1ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:39.791676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1eca4a-6d8a-4c6f-921c-97ac51cf1ec1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:39.803474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1091/2142] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:39.807570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c116d-0fa4-42fc-9180-62d18300e778 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:39.818053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c116d-0fa4-42fc-9180-62d18300e778 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:39.822979 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157e940-e0f6-4f7e-8191-c8944da14275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:39.823565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157e940-e0f6-4f7e-8191-c8944da14275 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:39.823839 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157e940-e0f6-4f7e-8191-c8944da14275 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:39.824106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157e940-e0f6-4f7e-8191-c8944da14275 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:39.824732 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157e940-e0f6-4f7e-8191-c8944da14275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:39.826367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3157e940-e0f6-4f7e-8191-c8944da14275 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:39.836258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1052/2143] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:39.840573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eca4a-6d8a-4c6f-921c-97ac51cf1ec1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:39.850454 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1eca4a-6d8a-4c6f-921c-97ac51cf1ec1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:39.855306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8af5ab-c0ce-4d49-84f5-a97139736453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:39.856505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8af5ab-c0ce-4d49-84f5-a97139736453 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:39.856505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8af5ab-c0ce-4d49-84f5-a97139736453 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:39.856505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8af5ab-c0ce-4d49-84f5-a97139736453 admin admin] PATH_INFO: `/v3/projects/429462c6a425456780bf3de30a7cedaa/users/343fe7c0c38246e78c76a122041abc4f/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:39.871885 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8af5ab-c0ce-4d49-84f5-a97139736453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=429462c6a425456780bf3de30a7cedaa, user_id=343fe7c0c38246e78c76a122041abc4f, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:39.874582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8af5ab-c0ce-4d49-84f5-a97139736453 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:39.950137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157e940-e0f6-4f7e-8191-c8944da14275 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:39.965048 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-3157e940-e0f6-4f7e-8191-c8944da14275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:39.982290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3157e940-e0f6-4f7e-8191-c8944da14275 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:39.986906 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33e5113-a228-4f45-be55-e7bb67788a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:39.988186 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e5113-a228-4f45-be55-e7bb67788a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:39.988466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e5113-a228-4f45-be55-e7bb67788a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:39.988766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33e5113-a228-4f45-be55-e7bb67788a36 admin admin] PATH_INFO: `/v3/projects/ea79258eb56c40fea9bf19e4c8fff327` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:39.989942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e5113-a228-4f45-be55-e7bb67788a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea79258eb56c40fea9bf19e4c8fff327)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:39.991961 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33e5113-a228-4f45-be55-e7bb67788a36 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:40.041865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1092/2144] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:57:39 2022] PUT /identity/v3/projects/429462c6a425456780bf3de30a7cedaa/users/343fe7c0c38246e78c76a122041abc4f/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:40.053233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ae5c8-1748-4840-af59-bb563200ea6d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:40.053515 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ae5c8-1748-4840-af59-bb563200ea6d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:40.053789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792ae5c8-1748-4840-af59-bb563200ea6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:40.097262 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-792ae5c8-1748-4840-af59-bb563200ea6d None None] MFA Rules not processed for user `343fe7c0c38246e78c76a122041abc4f`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:40.099253 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-792ae5c8-1748-4840-af59-bb563200ea6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:40.117364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d33e5113-a228-4f45-be55-e7bb67788a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea79258eb56c40fea9bf19e4c8fff327', 'request_id': 'req-d33e5113-a228-4f45-be55-e7bb67788a36'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:40.137175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d33e5113-a228-4f45-be55-e7bb67788a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea79258eb56c40fea9bf19e4c8fff327', 'request_id': 'req-d33e5113-a228-4f45-be55-e7bb67788a36'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:40.158804 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d33e5113-a228-4f45-be55-e7bb67788a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea79258eb56c40fea9bf19e4c8fff327', 'request_id': 'req-d33e5113-a228-4f45-be55-e7bb67788a36'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:40.227025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1093/2145] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:57:40.239502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ae5c8-1748-4840-af59-bb563200ea6d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:40.240918 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-792ae5c8-1748-4840-af59-bb563200ea6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:40.295023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:40.308367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d33e5113-a228-4f45-be55-e7bb67788a36 admin admin] The token cache is being invalidate because project ea79258eb56c40fea9bf19e4c8fff327 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:57:40.308827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d33e5113-a228-4f45-be55-e7bb67788a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:40.309610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d33e5113-a228-4f45-be55-e7bb67788a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:40.312166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1053/2146] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:57:39 2022] DELETE /identity/v3/projects/ea79258eb56c40fea9bf19e4c8fff327 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:40.316751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31daf6d1-29a6-4095-bbb2-1bcbb89291df None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:40.317164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31daf6d1-29a6-4095-bbb2-1bcbb89291df None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:40.317829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31daf6d1-29a6-4095-bbb2-1bcbb89291df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:40.342548 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792ae5c8-1748-4840-af59-bb563200ea6d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:40.343278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-31daf6d1-29a6-4095-bbb2-1bcbb89291df None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:40.344594 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-31daf6d1-29a6-4095-bbb2-1bcbb89291df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:40.354617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1546371-ef72-466c-828e-5a37bf1b23b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:40.354617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1546371-ef72-466c-828e-5a37bf1b23b6 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:40.354617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1546371-ef72-466c-828e-5a37bf1b23b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:40.354617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1546371-ef72-466c-828e-5a37bf1b23b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:40.355743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d1546371-ef72-466c-828e-5a37bf1b23b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:40.389297 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1546371-ef72-466c-828e-5a37bf1b23b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:40.389297 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1546371-ef72-466c-828e-5a37bf1b23b6 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:40.397997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1094/2147] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:40.437508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1054/2148] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:57:40.447098 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d425768-39a4-4bd2-abe6-7c3998e8eb76 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:40.447634 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d425768-39a4-4bd2-abe6-7c3998e8eb76 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:40.447906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d425768-39a4-4bd2-abe6-7c3998e8eb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:40.475602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1d425768-39a4-4bd2-abe6-7c3998e8eb76 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:40.477034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1d425768-39a4-4bd2-abe6-7c3998e8eb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:40.485195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1095/2149] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:40.490126 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31daf6d1-29a6-4095-bbb2-1bcbb89291df None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:40.499871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31daf6d1-29a6-4095-bbb2-1bcbb89291df None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:40.504619 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35528f86-c4a4-4363-a703-78e9474ed16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:40.505210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35528f86-c4a4-4363-a703-78e9474ed16f admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:40.505483 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35528f86-c4a4-4363-a703-78e9474ed16f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:40.505750 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35528f86-c4a4-4363-a703-78e9474ed16f admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:40.506882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35528f86-c4a4-4363-a703-78e9474ed16f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:57:40.508236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35528f86-c4a4-4363-a703-78e9474ed16f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:40.509839 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35528f86-c4a4-4363-a703-78e9474ed16f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:40.522408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1055/2150] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:57:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:57:40.526809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d425768-39a4-4bd2-abe6-7c3998e8eb76 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:40.539781 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d425768-39a4-4bd2-abe6-7c3998e8eb76 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:40.545464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619168b1-b5f1-410a-954b-26258c461208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:40.546086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619168b1-b5f1-410a-954b-26258c461208 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:40.546361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619168b1-b5f1-410a-954b-26258c461208 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:40.547012 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619168b1-b5f1-410a-954b-26258c461208 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:40.547653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619168b1-b5f1-410a-954b-26258c461208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:40.549699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619168b1-b5f1-410a-954b-26258c461208 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:40.607350 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:40.740676 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1096/2151] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:57:40 2022] POST /identity/v3/projects => generated 375 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:40.744747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35528f86-c4a4-4363-a703-78e9474ed16f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:40.754716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35528f86-c4a4-4363-a703-78e9474ed16f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:40.811969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1e9d6-03b2-437f-9c32-9ba22deb0585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:40.812613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1e9d6-03b2-437f-9c32-9ba22deb0585 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:40.812828 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1e9d6-03b2-437f-9c32-9ba22deb0585 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:40.813093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1e9d6-03b2-437f-9c32-9ba22deb0585 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:40.813710 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1e9d6-03b2-437f-9c32-9ba22deb0585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:40.815663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1e9d6-03b2-437f-9c32-9ba22deb0585 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:40.910276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1056/2152] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:57:40 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:40.914721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619168b1-b5f1-410a-954b-26258c461208 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:40.924908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619168b1-b5f1-410a-954b-26258c461208 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:40.929510 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00894450-86b5-41ee-ac49-3e272ff50e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:40.930145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00894450-86b5-41ee-ac49-3e272ff50e1e admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:40.930428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00894450-86b5-41ee-ac49-3e272ff50e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:40.930691 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00894450-86b5-41ee-ac49-3e272ff50e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:40.931339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00894450-86b5-41ee-ac49-3e272ff50e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:40.932936 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00894450-86b5-41ee-ac49-3e272ff50e1e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:40.943155 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1097/2153] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:40.951327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1e9d6-03b2-437f-9c32-9ba22deb0585 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:40.961860 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1e9d6-03b2-437f-9c32-9ba22deb0585 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:40.966689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fb4eb-0952-476a-9a5e-5096def41f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:40.967287 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fb4eb-0952-476a-9a5e-5096def41f9a admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:40.967560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fb4eb-0952-476a-9a5e-5096def41f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:40.967826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fb4eb-0952-476a-9a5e-5096def41f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:40.968485 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fb4eb-0952-476a-9a5e-5096def41f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:40.970155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fb4eb-0952-476a-9a5e-5096def41f9a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:40.981075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1057/2154] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:40.986049 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00894450-86b5-41ee-ac49-3e272ff50e1e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:40.995989 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00894450-86b5-41ee-ac49-3e272ff50e1e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:41.000734 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ace41-d1eb-4f1f-a922-4a0087816333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:41.001339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ace41-d1eb-4f1f-a922-4a0087816333 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:41.001613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ace41-d1eb-4f1f-a922-4a0087816333 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:41.001963 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ace41-d1eb-4f1f-a922-4a0087816333 admin admin] PATH_INFO: `/v3/projects/8d42d6edc5104db5ab91563fc5ea5728/users/9f3ac862e3fb4864aa0471f0aea14d20/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:41.017170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ace41-d1eb-4f1f-a922-4a0087816333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d42d6edc5104db5ab91563fc5ea5728, user_id=9f3ac862e3fb4864aa0471f0aea14d20, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:41.020526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ace41-d1eb-4f1f-a922-4a0087816333 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:41.165323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1098/2155] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:57:40 2022] PUT /identity/v3/projects/8d42d6edc5104db5ab91563fc5ea5728/users/9f3ac862e3fb4864aa0471f0aea14d20/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 19:57:41.175511 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c556cee-1dfe-497e-83f2-d604ab6be96a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:41.175795 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c556cee-1dfe-497e-83f2-d604ab6be96a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:41.176552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c556cee-1dfe-497e-83f2-d604ab6be96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:41.213233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-5c556cee-1dfe-497e-83f2-d604ab6be96a None None] MFA Rules not processed for user `9f3ac862e3fb4864aa0471f0aea14d20`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:41.214514 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-5c556cee-1dfe-497e-83f2-d604ab6be96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:41.343695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1058/2156] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:41.353406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ace41-d1eb-4f1f-a922-4a0087816333 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:41.355253 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ad0ace41-d1eb-4f1f-a922-4a0087816333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:41.446204 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ace41-d1eb-4f1f-a922-4a0087816333 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:41.450227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385fdd6-b74f-47b6-a7f8-99e7257e6a79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:41.450828 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385fdd6-b74f-47b6-a7f8-99e7257e6a79 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:41.451104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385fdd6-b74f-47b6-a7f8-99e7257e6a79 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:41.451374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385fdd6-b74f-47b6-a7f8-99e7257e6a79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:41.460669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385fdd6-b74f-47b6-a7f8-99e7257e6a79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:41.462318 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385fdd6-b74f-47b6-a7f8-99e7257e6a79 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:41.472184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1099/2157] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:57:42.214584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:42.808881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:42.869349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c556cee-1dfe-497e-83f2-d604ab6be96a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:42.870827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-5c556cee-1dfe-497e-83f2-d604ab6be96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:42.958745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c556cee-1dfe-497e-83f2-d604ab6be96a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:42.965019 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edc916-a114-4e63-9e3f-001c940e45e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:42.966704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edc916-a114-4e63-9e3f-001c940e45e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:42.967057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edc916-a114-4e63-9e3f-001c940e45e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:42.967401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06edc916-a114-4e63-9e3f-001c940e45e3 admin admin] PATH_INFO: `/v3/users/8136dabd3c184f16873c64459f4ad312` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:42.968947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edc916-a114-4e63-9e3f-001c940e45e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8136dabd3c184f16873c64459f4ad312)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:42.971989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06edc916-a114-4e63-9e3f-001c940e45e3 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:43.201218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-06edc916-a114-4e63-9e3f-001c940e45e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8136dabd3c184f16873c64459f4ad312'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:43.213510 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-06edc916-a114-4e63-9e3f-001c940e45e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8136dabd3c184f16873c64459f4ad312'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:43.288008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-06edc916-a114-4e63-9e3f-001c940e45e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8136dabd3c184f16873c64459f4ad312'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:43.294743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-06edc916-a114-4e63-9e3f-001c940e45e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8136dabd3c184f16873c64459f4ad312'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:43.308888 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-06edc916-a114-4e63-9e3f-001c940e45e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8136dabd3c184f16873c64459f4ad312'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:43.319580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1059/2158] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:57:42 2022] DELETE /identity/v3/users/8136dabd3c184f16873c64459f4ad312 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:43.323995 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8ba72-de32-4aee-9b4b-3386f728723d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:43.324474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8ba72-de32-4aee-9b4b-3386f728723d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:43.324905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f8ba72-de32-4aee-9b4b-3386f728723d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:43.347144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f6f8ba72-de32-4aee-9b4b-3386f728723d None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:43.348959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f6f8ba72-de32-4aee-9b4b-3386f728723d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:43.414273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1100/2159] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:43.425228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edc916-a114-4e63-9e3f-001c940e45e3 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:43.426689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-06edc916-a114-4e63-9e3f-001c940e45e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:43.488324 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06edc916-a114-4e63-9e3f-001c940e45e3 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:43.492071 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1eecb6-4f40-41fd-a41f-ad875c425182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:43.492683 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1eecb6-4f40-41fd-a41f-ad875c425182 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:43.493006 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1eecb6-4f40-41fd-a41f-ad875c425182 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:43.493289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1eecb6-4f40-41fd-a41f-ad875c425182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:43.502506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1eecb6-4f40-41fd-a41f-ad875c425182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:43.504434 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1eecb6-4f40-41fd-a41f-ad875c425182 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:43.515040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1060/2160] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:44.121272 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f8ba72-de32-4aee-9b4b-3386f728723d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:44.122878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f6f8ba72-de32-4aee-9b4b-3386f728723d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:44.134994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f8ba72-de32-4aee-9b4b-3386f728723d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:44.140454 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626271a4-a01b-4d03-8477-cea5a3de8c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:44.141351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626271a4-a01b-4d03-8477-cea5a3de8c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:44.141878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626271a4-a01b-4d03-8477-cea5a3de8c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:44.142221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626271a4-a01b-4d03-8477-cea5a3de8c57 admin admin] PATH_INFO: `/v3/projects/d1ec7e8608044db896133a6b38946299` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:44.143468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626271a4-a01b-4d03-8477-cea5a3de8c57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1ec7e8608044db896133a6b38946299)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:44.145644 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626271a4-a01b-4d03-8477-cea5a3de8c57 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:44.302818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-626271a4-a01b-4d03-8477-cea5a3de8c57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1ec7e8608044db896133a6b38946299', 'request_id': 'req-626271a4-a01b-4d03-8477-cea5a3de8c57'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:44.433218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-626271a4-a01b-4d03-8477-cea5a3de8c57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1ec7e8608044db896133a6b38946299', 'request_id': 'req-626271a4-a01b-4d03-8477-cea5a3de8c57'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:44.441403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-626271a4-a01b-4d03-8477-cea5a3de8c57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1ec7e8608044db896133a6b38946299', 'request_id': 'req-626271a4-a01b-4d03-8477-cea5a3de8c57'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:44.452745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-626271a4-a01b-4d03-8477-cea5a3de8c57 admin admin] The token cache is being invalidate because project d1ec7e8608044db896133a6b38946299 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:57:44.454319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-626271a4-a01b-4d03-8477-cea5a3de8c57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:44.454319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-626271a4-a01b-4d03-8477-cea5a3de8c57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:44.458906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1101/2161] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:57:44 2022] DELETE /identity/v3/projects/d1ec7e8608044db896133a6b38946299 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:44.460986 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea021f3e-c047-4b0a-8f7f-6e7292b3a458 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:44.460986 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea021f3e-c047-4b0a-8f7f-6e7292b3a458 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:44.461554 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea021f3e-c047-4b0a-8f7f-6e7292b3a458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:44.486091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-ea021f3e-c047-4b0a-8f7f-6e7292b3a458 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:44.487601 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ea021f3e-c047-4b0a-8f7f-6e7292b3a458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:44.549963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1061/2162] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:44.557660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd25d3-644a-49ed-91ed-c26693780e52 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:44.557960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd25d3-644a-49ed-91ed-c26693780e52 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:44.558288 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd25d3-644a-49ed-91ed-c26693780e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:44.578317 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-18cd25d3-644a-49ed-91ed-c26693780e52 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:44.579532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-18cd25d3-644a-49ed-91ed-c26693780e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:44.588773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1102/2163] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:44.593152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea021f3e-c047-4b0a-8f7f-6e7292b3a458 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:44.603792 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea021f3e-c047-4b0a-8f7f-6e7292b3a458 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:44.608564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14402d29-6e69-4849-b5f5-25dd6cb5eed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:44.610291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14402d29-6e69-4849-b5f5-25dd6cb5eed5 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:44.610291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14402d29-6e69-4849-b5f5-25dd6cb5eed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:44.610291 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14402d29-6e69-4849-b5f5-25dd6cb5eed5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:44.610630 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14402d29-6e69-4849-b5f5-25dd6cb5eed5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:57:44.610762 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14402d29-6e69-4849-b5f5-25dd6cb5eed5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:44.612343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14402d29-6e69-4849-b5f5-25dd6cb5eed5 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:44.624290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1062/2164] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:57:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:57:44.629321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd25d3-644a-49ed-91ed-c26693780e52 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:44.638403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd25d3-644a-49ed-91ed-c26693780e52 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:44.643018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59089470-70fe-4fb5-952b-e973678cb685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:44.644192 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59089470-70fe-4fb5-952b-e973678cb685 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:44.644467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59089470-70fe-4fb5-952b-e973678cb685 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:44.644738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59089470-70fe-4fb5-952b-e973678cb685 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:44.645358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59089470-70fe-4fb5-952b-e973678cb685 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:44.647390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59089470-70fe-4fb5-952b-e973678cb685 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:44.755002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1103/2165] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:57:44 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:44.756125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14402d29-6e69-4849-b5f5-25dd6cb5eed5 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:44.765893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14402d29-6e69-4849-b5f5-25dd6cb5eed5 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:44.821773 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478ee32-68ea-4130-a37c-441e063d2cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:44.822424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478ee32-68ea-4130-a37c-441e063d2cdd admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:44.823022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478ee32-68ea-4130-a37c-441e063d2cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:44.823341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a478ee32-68ea-4130-a37c-441e063d2cdd admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:44.823949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478ee32-68ea-4130-a37c-441e063d2cdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:44.825879 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a478ee32-68ea-4130-a37c-441e063d2cdd admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:45.157498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1063/2166] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:57:44 2022] POST /identity/v3/users => generated 286 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:57:45.158404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:45.167535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59089470-70fe-4fb5-952b-e973678cb685 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:45.179387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59089470-70fe-4fb5-952b-e973678cb685 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:45.184345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8de407-d3aa-449d-8652-924f20ef107b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:45.184965 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de407-d3aa-449d-8652-924f20ef107b admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:45.185238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de407-d3aa-449d-8652-924f20ef107b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:45.185530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de407-d3aa-449d-8652-924f20ef107b admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:45.186244 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8de407-d3aa-449d-8652-924f20ef107b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:45.187938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8de407-d3aa-449d-8652-924f20ef107b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:45.199964 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1104/2167] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:45.203868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478ee32-68ea-4130-a37c-441e063d2cdd admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:45.214926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a478ee32-68ea-4130-a37c-441e063d2cdd admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:45.220073 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab0c349-1432-4dd6-870e-2b66e39b64a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:45.220696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0c349-1432-4dd6-870e-2b66e39b64a0 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:45.220993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0c349-1432-4dd6-870e-2b66e39b64a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:45.221289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab0c349-1432-4dd6-870e-2b66e39b64a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:45.222000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0c349-1432-4dd6-870e-2b66e39b64a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:45.223572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab0c349-1432-4dd6-870e-2b66e39b64a0 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:45.235070 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1064/2168] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:45.240102 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8de407-d3aa-449d-8652-924f20ef107b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:45.250728 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8de407-d3aa-449d-8652-924f20ef107b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:45.330628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb574b-ae18-482e-ab26-aa5ad0e91d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:45.330628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb574b-ae18-482e-ab26-aa5ad0e91d35 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:45.330628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb574b-ae18-482e-ab26-aa5ad0e91d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:45.330628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb574b-ae18-482e-ab26-aa5ad0e91d35 admin admin] PATH_INFO: `/v3/projects/1ee6a0358ff94f47b892862d44c3b17c/users/c0a1e3e35e814808ae7540d35ba19d0d/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:45.332284 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb574b-ae18-482e-ab26-aa5ad0e91d35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ee6a0358ff94f47b892862d44c3b17c, user_id=c0a1e3e35e814808ae7540d35ba19d0d, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:45.332284 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb574b-ae18-482e-ab26-aa5ad0e91d35 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:45.765568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:46.250405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1105/2169] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:57:45 2022] PUT /identity/v3/projects/1ee6a0358ff94f47b892862d44c3b17c/users/c0a1e3e35e814808ae7540d35ba19d0d/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 1011 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:46.261451 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9111ecb-b1fd-491b-bc94-7a93274e6f3c None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:46.261926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9111ecb-b1fd-491b-bc94-7a93274e6f3c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:46.262124 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9111ecb-b1fd-491b-bc94-7a93274e6f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:46.292160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b9111ecb-b1fd-491b-bc94-7a93274e6f3c None None] MFA Rules not processed for user `c0a1e3e35e814808ae7540d35ba19d0d`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:46.293353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b9111ecb-b1fd-491b-bc94-7a93274e6f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:46.366045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1065/2170] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:46.377575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb574b-ae18-482e-ab26-aa5ad0e91d35 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:46.380035 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-93fb574b-ae18-482e-ab26-aa5ad0e91d35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:46.445076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb574b-ae18-482e-ab26-aa5ad0e91d35 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:46.449050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e049e-2935-4c53-a686-cf3df2a02e57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:46.449655 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e049e-2935-4c53-a686-cf3df2a02e57 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:46.449971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e049e-2935-4c53-a686-cf3df2a02e57 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:46.450240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e049e-2935-4c53-a686-cf3df2a02e57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:46.459184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e049e-2935-4c53-a686-cf3df2a02e57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:46.461824 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e049e-2935-4c53-a686-cf3df2a02e57 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:46.469858 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1106/2171] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:46 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:46.497178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9111ecb-b1fd-491b-bc94-7a93274e6f3c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:46.568235 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9111ecb-b1fd-491b-bc94-7a93274e6f3c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:46.573218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1256925-5913-4c07-9d40-e51a23e6b07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:46.573961 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1256925-5913-4c07-9d40-e51a23e6b07f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:46.574240 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1256925-5913-4c07-9d40-e51a23e6b07f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:46.574781 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1256925-5913-4c07-9d40-e51a23e6b07f admin admin] PATH_INFO: `/v3/users/c0a1e3e35e814808ae7540d35ba19d0d` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:46.576056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1256925-5913-4c07-9d40-e51a23e6b07f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0a1e3e35e814808ae7540d35ba19d0d)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:46.578021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1256925-5913-4c07-9d40-e51a23e6b07f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:46.827127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-b1256925-5913-4c07-9d40-e51a23e6b07f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0a1e3e35e814808ae7540d35ba19d0d'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:46.838796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-b1256925-5913-4c07-9d40-e51a23e6b07f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0a1e3e35e814808ae7540d35ba19d0d'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:46.929722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-b1256925-5913-4c07-9d40-e51a23e6b07f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0a1e3e35e814808ae7540d35ba19d0d'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:46.930444 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-b1256925-5913-4c07-9d40-e51a23e6b07f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0a1e3e35e814808ae7540d35ba19d0d'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:46.938000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-b1256925-5913-4c07-9d40-e51a23e6b07f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0a1e3e35e814808ae7540d35ba19d0d'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:46.943960 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1066/2172] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:57:46 2022] DELETE /identity/v3/users/c0a1e3e35e814808ae7540d35ba19d0d => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:46.948971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f56dd-d877-4732-889d-0b87d31a81a6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:46.949261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f56dd-d877-4732-889d-0b87d31a81a6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:46.949546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f56dd-d877-4732-889d-0b87d31a81a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:46.969310 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-176f56dd-d877-4732-889d-0b87d31a81a6 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:46.973296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-176f56dd-d877-4732-889d-0b87d31a81a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:47.038760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1107/2173] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:57:47.048562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1256925-5913-4c07-9d40-e51a23e6b07f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:47.050686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b1256925-5913-4c07-9d40-e51a23e6b07f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:47.110197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1256925-5913-4c07-9d40-e51a23e6b07f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:47.114391 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74605a3c-04d6-46cf-ae86-f45928644bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:47.114983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74605a3c-04d6-46cf-ae86-f45928644bcf service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:47.115255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74605a3c-04d6-46cf-ae86-f45928644bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:47.115519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74605a3c-04d6-46cf-ae86-f45928644bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:47.124176 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74605a3c-04d6-46cf-ae86-f45928644bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:47.125592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74605a3c-04d6-46cf-ae86-f45928644bcf service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:47.136360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1067/2174] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:47.510532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:47.636440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f56dd-d877-4732-889d-0b87d31a81a6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:47.637873 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-176f56dd-d877-4732-889d-0b87d31a81a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:47.650794 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f56dd-d877-4732-889d-0b87d31a81a6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:47.654083 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:47.656015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db675d25-327d-4551-a017-2e8c74f03f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:47.656652 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db675d25-327d-4551-a017-2e8c74f03f68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:47.657048 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db675d25-327d-4551-a017-2e8c74f03f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:47.657500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db675d25-327d-4551-a017-2e8c74f03f68 admin admin] PATH_INFO: `/v3/projects/1ee6a0358ff94f47b892862d44c3b17c` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:47.658801 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db675d25-327d-4551-a017-2e8c74f03f68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ee6a0358ff94f47b892862d44c3b17c)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:47.660894 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db675d25-327d-4551-a017-2e8c74f03f68 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:47.784104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-db675d25-327d-4551-a017-2e8c74f03f68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ee6a0358ff94f47b892862d44c3b17c', 'request_id': 'req-db675d25-327d-4551-a017-2e8c74f03f68'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:48.000351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-db675d25-327d-4551-a017-2e8c74f03f68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ee6a0358ff94f47b892862d44c3b17c', 'request_id': 'req-db675d25-327d-4551-a017-2e8c74f03f68'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:48.000351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-db675d25-327d-4551-a017-2e8c74f03f68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ee6a0358ff94f47b892862d44c3b17c', 'request_id': 'req-db675d25-327d-4551-a017-2e8c74f03f68'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:48.000351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-db675d25-327d-4551-a017-2e8c74f03f68 admin admin] The token cache is being invalidate because project 1ee6a0358ff94f47b892862d44c3b17c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:57:48.000351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-db675d25-327d-4551-a017-2e8c74f03f68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:48.001495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-db675d25-327d-4551-a017-2e8c74f03f68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:57:48.001495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1108/2175] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:57:47 2022] DELETE /identity/v3/projects/1ee6a0358ff94f47b892862d44c3b17c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:48.001686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be332abe-16ed-4a7f-bc37-5a8ef45dfa2a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:48.001686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be332abe-16ed-4a7f-bc37-5a8ef45dfa2a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:48.001686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be332abe-16ed-4a7f-bc37-5a8ef45dfa2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:48.001686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-be332abe-16ed-4a7f-bc37-5a8ef45dfa2a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:48.001686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-be332abe-16ed-4a7f-bc37-5a8ef45dfa2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:48.027044 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1068/2176] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:48.036079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30012300-c74e-4359-b66a-f0ad952fb54e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:48.036079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30012300-c74e-4359-b66a-f0ad952fb54e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:48.036079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30012300-c74e-4359-b66a-f0ad952fb54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:48.059340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-30012300-c74e-4359-b66a-f0ad952fb54e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:48.060743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-30012300-c74e-4359-b66a-f0ad952fb54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:48.070697 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1109/2177] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:48.075209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be332abe-16ed-4a7f-bc37-5a8ef45dfa2a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:48.085036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be332abe-16ed-4a7f-bc37-5a8ef45dfa2a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:48.090242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b8128-73c5-42e2-8325-c9a636a56bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:48.093408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b8128-73c5-42e2-8325-c9a636a56bd2 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:48.093408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b8128-73c5-42e2-8325-c9a636a56bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:48.093408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56b8128-73c5-42e2-8325-c9a636a56bd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:48.093408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b8128-73c5-42e2-8325-c9a636a56bd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:57:48.093408 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b8128-73c5-42e2-8325-c9a636a56bd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:48.093919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56b8128-73c5-42e2-8325-c9a636a56bd2 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:48.106026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1069/2178] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:57:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:57:48.110518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30012300-c74e-4359-b66a-f0ad952fb54e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:48.120280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30012300-c74e-4359-b66a-f0ad952fb54e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:48.124908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99178104-5adc-442e-bf13-3cac33285bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:48.126737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99178104-5adc-442e-bf13-3cac33285bfa admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:48.126737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99178104-5adc-442e-bf13-3cac33285bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:48.126737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99178104-5adc-442e-bf13-3cac33285bfa admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:48.126737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99178104-5adc-442e-bf13-3cac33285bfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:48.129273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99178104-5adc-442e-bf13-3cac33285bfa admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:48.263427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1110/2179] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:57:48 2022] POST /identity/v3/projects => generated 381 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:57:48.269872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b8128-73c5-42e2-8325-c9a636a56bd2 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:48.279703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56b8128-73c5-42e2-8325-c9a636a56bd2 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:48.338473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a014f-0547-4e16-b173-1eafeb0bff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:48.339080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a014f-0547-4e16-b173-1eafeb0bff59 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:48.339362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a014f-0547-4e16-b173-1eafeb0bff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:48.339634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a014f-0547-4e16-b173-1eafeb0bff59 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:48.340311 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a014f-0547-4e16-b173-1eafeb0bff59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:48.342292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a014f-0547-4e16-b173-1eafeb0bff59 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:48.442440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1070/2180] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:57:48 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:57:48.446807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99178104-5adc-442e-bf13-3cac33285bfa admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:48.457618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99178104-5adc-442e-bf13-3cac33285bfa admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:48.468188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1b34e-ba8a-461d-9829-5e58d62bbe3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:48.468188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1b34e-ba8a-461d-9829-5e58d62bbe3d admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:48.468188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1b34e-ba8a-461d-9829-5e58d62bbe3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:48.468188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1b34e-ba8a-461d-9829-5e58d62bbe3d admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:48.468573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1b34e-ba8a-461d-9829-5e58d62bbe3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:48.468573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1b34e-ba8a-461d-9829-5e58d62bbe3d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:48.476545 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1111/2181] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:48.480956 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a014f-0547-4e16-b173-1eafeb0bff59 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:48.491615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a014f-0547-4e16-b173-1eafeb0bff59 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:48.504781 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aff1855-4f5b-4713-bf41-431bf746731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:48.504781 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff1855-4f5b-4713-bf41-431bf746731b admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:48.504781 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff1855-4f5b-4713-bf41-431bf746731b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:48.504781 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aff1855-4f5b-4713-bf41-431bf746731b admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:48.505143 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff1855-4f5b-4713-bf41-431bf746731b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:48.505143 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aff1855-4f5b-4713-bf41-431bf746731b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:48.511594 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1071/2182] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:57:48.515585 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1b34e-ba8a-461d-9829-5e58d62bbe3d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:48.526607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1b34e-ba8a-461d-9829-5e58d62bbe3d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:48.542920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2da75-416b-4dc8-838c-2670ce65f8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:48.542920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2da75-416b-4dc8-838c-2670ce65f8de admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:48.542920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2da75-416b-4dc8-838c-2670ce65f8de admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:48.542920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c2da75-416b-4dc8-838c-2670ce65f8de admin admin] PATH_INFO: `/v3/projects/bb2e9dd680194d6e9b861fed73188ba6/users/a66d0ec3a156469980551f9bc9034d4e/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:48.551390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2da75-416b-4dc8-838c-2670ce65f8de admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb2e9dd680194d6e9b861fed73188ba6, user_id=a66d0ec3a156469980551f9bc9034d4e, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:48.554056 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c2da75-416b-4dc8-838c-2670ce65f8de admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:48.640476 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1112/2183] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:57:48 2022] PUT /identity/v3/projects/bb2e9dd680194d6e9b861fed73188ba6/users/a66d0ec3a156469980551f9bc9034d4e/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:57:48.646270 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b037-e3af-4054-a31a-9abcc148038c None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:48.646655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b037-e3af-4054-a31a-9abcc148038c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:48.646994 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9949b037-e3af-4054-a31a-9abcc148038c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:48.667316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-9949b037-e3af-4054-a31a-9abcc148038c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:48.668522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-9949b037-e3af-4054-a31a-9abcc148038c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:48.742880 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1072/2184] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:48.750086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2da75-416b-4dc8-838c-2670ce65f8de admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:48.751857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b5c2da75-416b-4dc8-838c-2670ce65f8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:48.836975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c2da75-416b-4dc8-838c-2670ce65f8de admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:48.838351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269cd3-6656-44ce-900c-fdbb983039ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:48.838919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269cd3-6656-44ce-900c-fdbb983039ac service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:48.839195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269cd3-6656-44ce-900c-fdbb983039ac service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:48.839463 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269cd3-6656-44ce-900c-fdbb983039ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:48.849755 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44269cd3-6656-44ce-900c-fdbb983039ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:48.851202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44269cd3-6656-44ce-900c-fdbb983039ac service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:48.861717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1113/2185] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:49.705024 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:49.973896 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:50.522645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6e8e0-6311-4d0b-8ec6-bb1eb1765ede None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:50.523766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6e8e0-6311-4d0b-8ec6-bb1eb1765ede None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:50.524048 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6e8e0-6311-4d0b-8ec6-bb1eb1765ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:50.555707 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-05a6e8e0-6311-4d0b-8ec6-bb1eb1765ede None None] MFA Rules not processed for user `a66d0ec3a156469980551f9bc9034d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:50.556330 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-05a6e8e0-6311-4d0b-8ec6-bb1eb1765ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:50.630325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1073/2186] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:50.640754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269cd3-6656-44ce-900c-fdbb983039ac service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:50.642324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-44269cd3-6656-44ce-900c-fdbb983039ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:50.712080 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269cd3-6656-44ce-900c-fdbb983039ac service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:50.717108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb38d72-3ed3-4bde-82d9-ef92e424ae7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:50.717893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb38d72-3ed3-4bde-82d9-ef92e424ae7d service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:50.718257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb38d72-3ed3-4bde-82d9-ef92e424ae7d service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:50.718694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb38d72-3ed3-4bde-82d9-ef92e424ae7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:50.729635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb38d72-3ed3-4bde-82d9-ef92e424ae7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:50.731231 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb38d72-3ed3-4bde-82d9-ef92e424ae7d service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:50.741353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1114/2187] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:57:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:57:51.240283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:51.451064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:52.942977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:53.196557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46c69e-ce76-4d3f-aa78-bc06abe34ae9 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:53.196843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46c69e-ce76-4d3f-aa78-bc06abe34ae9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:53.197116 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46c69e-ce76-4d3f-aa78-bc06abe34ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:53.217920 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-ef46c69e-ce76-4d3f-aa78-bc06abe34ae9 None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:53.219122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ef46c69e-ce76-4d3f-aa78-bc06abe34ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:53.337080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1074/2188] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 19:57:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:53.342492 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:53.351614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb38d72-3ed3-4bde-82d9-ef92e424ae7d service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:57:53.353628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3bb38d72-3ed3-4bde-82d9-ef92e424ae7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:53.423920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb38d72-3ed3-4bde-82d9-ef92e424ae7d service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:57:53.429506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf07cd-0396-4028-8953-9fd1301ef280 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:57:53.444330 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf07cd-0396-4028-8953-9fd1301ef280 service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:53.444330 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf07cd-0396-4028-8953-9fd1301ef280 service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:53.444330 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf07cd-0396-4028-8953-9fd1301ef280 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:53.444330 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf07cd-0396-4028-8953-9fd1301ef280 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:57:53.444330 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf07cd-0396-4028-8953-9fd1301ef280 service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:57:53.449586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1115/2189] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 19:57:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 19:57:53.781197 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437b868-f3b0-40fd-b200-8814eb4b7371 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:57:53.788705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437b868-f3b0-40fd-b200-8814eb4b7371 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:57:53.788705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0437b868-f3b0-40fd-b200-8814eb4b7371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:57:53.806330 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-0437b868-f3b0-40fd-b200-8814eb4b7371 None None] MFA Rules not processed for user `a66d0ec3a156469980551f9bc9034d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:57:53.807876 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0437b868-f3b0-40fd-b200-8814eb4b7371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:57:53.838391 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1075/2190] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:57:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:57:55.085040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:55.233980 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:58.227963 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:58.419917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:57:59.295461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:58:08.515221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf07cd-0396-4028-8953-9fd1301ef280 service swift] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:08.526151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf07cd-0396-4028-8953-9fd1301ef280 service swift] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:08.530087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3712d339-297a-46c6-b5d6-9b8b27fc7c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:08.534582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712d339-297a-46c6-b5d6-9b8b27fc7c7d service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:08.534582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712d339-297a-46c6-b5d6-9b8b27fc7c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:08.534582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3712d339-297a-46c6-b5d6-9b8b27fc7c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:08.534582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3712d339-297a-46c6-b5d6-9b8b27fc7c7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:08.580936 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:58:08.597720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712d339-297a-46c6-b5d6-9b8b27fc7c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:08.599322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3712d339-297a-46c6-b5d6-9b8b27fc7c7d service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:08.609115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1116/2191] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:58:08 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:58:09.730903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=15 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:58:12.120054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437b868-f3b0-40fd-b200-8814eb4b7371 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:12.121566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0437b868-f3b0-40fd-b200-8814eb4b7371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:12.148890 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0437b868-f3b0-40fd-b200-8814eb4b7371 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:12.163748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f13f7-77f7-4557-ba28-d991183745a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:12.165234 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f13f7-77f7-4557-ba28-d991183745a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:12.166051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f13f7-77f7-4557-ba28-d991183745a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:12.167806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f13f7-77f7-4557-ba28-d991183745a9 admin admin] PATH_INFO: `/v3/users/9f3ac862e3fb4864aa0471f0aea14d20` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:12.170542 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f13f7-77f7-4557-ba28-d991183745a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f3ac862e3fb4864aa0471f0aea14d20)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:12.174635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f13f7-77f7-4557-ba28-d991183745a9 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:12.562320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0e5f13f7-77f7-4557-ba28-d991183745a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f3ac862e3fb4864aa0471f0aea14d20'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:12.587607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0e5f13f7-77f7-4557-ba28-d991183745a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f3ac862e3fb4864aa0471f0aea14d20'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:12.725697 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0e5f13f7-77f7-4557-ba28-d991183745a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f3ac862e3fb4864aa0471f0aea14d20'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:12.726394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0e5f13f7-77f7-4557-ba28-d991183745a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f3ac862e3fb4864aa0471f0aea14d20'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:12.742390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0e5f13f7-77f7-4557-ba28-d991183745a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f3ac862e3fb4864aa0471f0aea14d20'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:12.760918 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1076/2192] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:58:12 2022] DELETE /identity/v3/users/9f3ac862e3fb4864aa0471f0aea14d20 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:58:12.765901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff13df-90f5-4205-990f-2cce96620a7f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:12.766394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff13df-90f5-4205-990f-2cce96620a7f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:12.769918 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff13df-90f5-4205-990f-2cce96620a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:12.794959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5bff13df-90f5-4205-990f-2cce96620a7f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:58:12.796191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5bff13df-90f5-4205-990f-2cce96620a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:12.883759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1117/2193] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:58:12.899550 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f13f7-77f7-4557-ba28-d991183745a9 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:12.904336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0e5f13f7-77f7-4557-ba28-d991183745a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:13.040897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f13f7-77f7-4557-ba28-d991183745a9 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:13.044965 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c8086-8c34-4972-a689-a3d5c440d034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:13.045841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8086-8c34-4972-a689-a3d5c440d034 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:13.046026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8086-8c34-4972-a689-a3d5c440d034 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:13.046165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8086-8c34-4972-a689-a3d5c440d034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:13.056183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c8086-8c34-4972-a689-a3d5c440d034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:13.058103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c8086-8c34-4972-a689-a3d5c440d034 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:13.071228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1077/2194] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:58:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:58:13.737934 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff13df-90f5-4205-990f-2cce96620a7f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:13.739790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5bff13df-90f5-4205-990f-2cce96620a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:13.753678 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bff13df-90f5-4205-990f-2cce96620a7f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:13.759675 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:13.760522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:13.760999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:13.771443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9 admin admin] PATH_INFO: `/v3/projects/8d42d6edc5104db5ab91563fc5ea5728` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:13.774580 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d42d6edc5104db5ab91563fc5ea5728)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:13.775145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:13.930032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d42d6edc5104db5ab91563fc5ea5728', 'request_id': 'req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:14.005223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d42d6edc5104db5ab91563fc5ea5728', 'request_id': 'req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:14.012659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d42d6edc5104db5ab91563fc5ea5728', 'request_id': 'req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:14.024181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9 admin admin] The token cache is being invalidate because project 8d42d6edc5104db5ab91563fc5ea5728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:58:14.024470 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:14.025141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-aec2e3df-b4ca-4640-bc8a-d529bec35ef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:14.026670 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1118/2195] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:58:13 2022] DELETE /identity/v3/projects/8d42d6edc5104db5ab91563fc5ea5728 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:58:14.971844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b502ce-6b3a-4711-807c-52b7da31a68c None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:14.973383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b502ce-6b3a-4711-807c-52b7da31a68c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:14.973766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b502ce-6b3a-4711-807c-52b7da31a68c None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:14.975378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1078/2196] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 19:58:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:58:14.992669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900db28-4303-46e6-8b52-5fa5d48e7bfb None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:14.993249 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900db28-4303-46e6-8b52-5fa5d48e7bfb None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:14.993355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4900db28-4303-46e6-8b52-5fa5d48e7bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:15.015264 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4900db28-4303-46e6-8b52-5fa5d48e7bfb None None] MFA Rules not processed for user `2a4284b59f804dc59674d621d87634bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:58:15.018798 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4900db28-4303-46e6-8b52-5fa5d48e7bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:15.118846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:58:15.135909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1119/2197] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 19:58:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:58:15.152672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b502ce-6b3a-4711-807c-52b7da31a68c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:15.154610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-29b502ce-6b3a-4711-807c-52b7da31a68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:15.257678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b502ce-6b3a-4711-807c-52b7da31a68c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:15.262011 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11261a8-2b6c-4110-96a6-41380d4770d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e021a00524341359bc4b5a800a9bbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:15.262762 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11261a8-2b6c-4110-96a6-41380d4770d8 service glance] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:15.263138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11261a8-2b6c-4110-96a6-41380d4770d8 service glance] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:15.263506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11261a8-2b6c-4110-96a6-41380d4770d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:15.278208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11261a8-2b6c-4110-96a6-41380d4770d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:15.279690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11261a8-2b6c-4110-96a6-41380d4770d8 service glance] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:15.294205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1079/2198] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:58:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:58:15.642441 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7c05a-f650-41b3-b6ce-0578f0128a42 None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:15.642680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7c05a-f650-41b3-b6ce-0578f0128a42 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:15.642964 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a7c05a-f650-41b3-b6ce-0578f0128a42 None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:15.645859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1120/2199] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 19:58:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 19:58:15.651385 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6d65c-a324-4b5f-8931-7c9804620cf5 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:15.651932 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6d65c-a324-4b5f-8931-7c9804620cf5 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:15.652221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb6d65c-a324-4b5f-8931-7c9804620cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:15.673879 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-1fb6d65c-a324-4b5f-8931-7c9804620cf5 None None] MFA Rules not processed for user `7103ff9544524393a8c7175998dcae3b`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:58:15.675119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1fb6d65c-a324-4b5f-8931-7c9804620cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:15.687932 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:58:15.739437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1080/2200] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 19:58:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:58:15.752966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7c05a-f650-41b3-b6ce-0578f0128a42 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:15.754978 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-28a7c05a-f650-41b3-b6ce-0578f0128a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:15.770512 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a7c05a-f650-41b3-b6ce-0578f0128a42 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:15.781029 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f78f0-f05e-4f0d-ba50-3c4e0283c009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:15.781990 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f78f0-f05e-4f0d-ba50-3c4e0283c009 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:15.782271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f78f0-f05e-4f0d-ba50-3c4e0283c009 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:15.782538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f78f0-f05e-4f0d-ba50-3c4e0283c009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:15.791117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f78f0-f05e-4f0d-ba50-3c4e0283c009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:15.792631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f78f0-f05e-4f0d-ba50-3c4e0283c009 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:15.804300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1121/2201] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:58:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:58:15.967655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c8338-7e4e-4596-a038-2c5177e46682 None None] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:15.967980 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c8338-7e4e-4596-a038-2c5177e46682 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:15.968493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c8338-7e4e-4596-a038-2c5177e46682 None None] PATH_INFO: `/` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:15.973857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1081/2202] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 19:58:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 19:58:15.980486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f25f0-610d-4b7a-b06a-f6ab19332f71 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:15.981098 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f25f0-610d-4b7a-b06a-f6ab19332f71 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:15.981374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f25f0-610d-4b7a-b06a-f6ab19332f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:16.012142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0b3f25f0-610d-4b7a-b06a-f6ab19332f71 None None] MFA Rules not processed for user `2a4284b59f804dc59674d621d87634bc`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:58:16.013438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0b3f25f0-610d-4b7a-b06a-f6ab19332f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:16.024314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1122/2203] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 19:58:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:58:16.034562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c8338-7e4e-4596-a038-2c5177e46682 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:16.052175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9c8338-7e4e-4596-a038-2c5177e46682 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:16.055647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23431e2-fee2-4bc2-9ad5-ed49fa238d7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:16.056635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23431e2-fee2-4bc2-9ad5-ed49fa238d7f service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:16.057201 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23431e2-fee2-4bc2-9ad5-ed49fa238d7f service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:16.057769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23431e2-fee2-4bc2-9ad5-ed49fa238d7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:16.084349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23431e2-fee2-4bc2-9ad5-ed49fa238d7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:16.086045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23431e2-fee2-4bc2-9ad5-ed49fa238d7f service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:16.101023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1082/2204] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:58:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:58:20.676177 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4accd-6e16-4197-956e-bf585d6edccf None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:20.676681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4accd-6e16-4197-956e-bf585d6edccf None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:20.676780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4accd-6e16-4197-956e-bf585d6edccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:20.696844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1dc4accd-6e16-4197-956e-bf585d6edccf None None] MFA Rules not processed for user `766dd5a650fc4855980250969854f08d`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:58:20.698351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1dc4accd-6e16-4197-956e-bf585d6edccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:20.775492 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1123/2205] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:58:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:58:24.115525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca79fb2-7f5d-44a0-83fb-1172c45ec2da None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:24.116067 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca79fb2-7f5d-44a0-83fb-1172c45ec2da None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:24.116297 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca79fb2-7f5d-44a0-83fb-1172c45ec2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:24.136187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-dca79fb2-7f5d-44a0-83fb-1172c45ec2da None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:58:24.137386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-dca79fb2-7f5d-44a0-83fb-1172c45ec2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:24.201157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1083/2206] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 19:58:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:58:24.214271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4accd-6e16-4197-956e-bf585d6edccf None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:24.215725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1dc4accd-6e16-4197-956e-bf585d6edccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:24.292056 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4accd-6e16-4197-956e-bf585d6edccf None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:24.297256 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40abfd6-7a09-4fc0-9728-0715de3bb520 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:24.297913 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40abfd6-7a09-4fc0-9728-0715de3bb520 service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:24.298188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40abfd6-7a09-4fc0-9728-0715de3bb520 service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:24.298451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40abfd6-7a09-4fc0-9728-0715de3bb520 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:24.307837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40abfd6-7a09-4fc0-9728-0715de3bb520 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:24.309561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40abfd6-7a09-4fc0-9728-0715de3bb520 service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:24.319181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1124/2207] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 19:58:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 19:58:34.194208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=46 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:58:34.314797 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=67 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:58:41.086165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca79fb2-7f5d-44a0-83fb-1172c45ec2da None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:41.087745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-dca79fb2-7f5d-44a0-83fb-1172c45ec2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:41.156554 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca79fb2-7f5d-44a0-83fb-1172c45ec2da None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:41.161597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62c619-bd49-43b8-8f0d-6aa7ea5491dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:41.162251 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62c619-bd49-43b8-8f0d-6aa7ea5491dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:41.162527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62c619-bd49-43b8-8f0d-6aa7ea5491dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:41.162795 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62c619-bd49-43b8-8f0d-6aa7ea5491dc admin admin] PATH_INFO: `/v3/users/343fe7c0c38246e78c76a122041abc4f` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:41.163908 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62c619-bd49-43b8-8f0d-6aa7ea5491dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=343fe7c0c38246e78c76a122041abc4f)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:41.165824 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62c619-bd49-43b8-8f0d-6aa7ea5491dc admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:41.420295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8a62c619-bd49-43b8-8f0d-6aa7ea5491dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '343fe7c0c38246e78c76a122041abc4f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:41.432349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8a62c619-bd49-43b8-8f0d-6aa7ea5491dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '343fe7c0c38246e78c76a122041abc4f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:41.568525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8a62c619-bd49-43b8-8f0d-6aa7ea5491dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '343fe7c0c38246e78c76a122041abc4f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:41.569300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8a62c619-bd49-43b8-8f0d-6aa7ea5491dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '343fe7c0c38246e78c76a122041abc4f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:41.577501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8a62c619-bd49-43b8-8f0d-6aa7ea5491dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '343fe7c0c38246e78c76a122041abc4f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:41.584169 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1084/2208] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:58:41 2022] DELETE /identity/v3/users/343fe7c0c38246e78c76a122041abc4f => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:58:41.588600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eba2a2-45b5-412b-a7b3-c3ce617bd424 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:41.589062 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eba2a2-45b5-412b-a7b3-c3ce617bd424 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:41.589157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eba2a2-45b5-412b-a7b3-c3ce617bd424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:41.608306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c4eba2a2-45b5-412b-a7b3-c3ce617bd424 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:58:41.609489 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c4eba2a2-45b5-412b-a7b3-c3ce617bd424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:41.669135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1125/2209] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:58:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:58:41.678281 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62c619-bd49-43b8-8f0d-6aa7ea5491dc admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:41.679726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8a62c619-bd49-43b8-8f0d-6aa7ea5491dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:41.738061 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62c619-bd49-43b8-8f0d-6aa7ea5491dc admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:41.742523 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4458efcc-a833-466f-9e0f-2d66c01589ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:41.743129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458efcc-a833-466f-9e0f-2d66c01589ad service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:41.743406 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458efcc-a833-466f-9e0f-2d66c01589ad service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:41.743808 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4458efcc-a833-466f-9e0f-2d66c01589ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:41.753152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458efcc-a833-466f-9e0f-2d66c01589ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:41.754708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4458efcc-a833-466f-9e0f-2d66c01589ad service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:41.767168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1085/2210] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:58:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:58:42.177686 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eba2a2-45b5-412b-a7b3-c3ce617bd424 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:42.179223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c4eba2a2-45b5-412b-a7b3-c3ce617bd424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:42.203117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eba2a2-45b5-412b-a7b3-c3ce617bd424 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:42.219654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ebc30-724f-4ab2-a492-6ee9974ac337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:42.219654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ebc30-724f-4ab2-a492-6ee9974ac337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:42.219654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ebc30-724f-4ab2-a492-6ee9974ac337 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:42.219654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ebc30-724f-4ab2-a492-6ee9974ac337 admin admin] PATH_INFO: `/v3/projects/429462c6a425456780bf3de30a7cedaa` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:42.219987 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ebc30-724f-4ab2-a492-6ee9974ac337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=429462c6a425456780bf3de30a7cedaa)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:42.219987 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9ebc30-724f-4ab2-a492-6ee9974ac337 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:42.338280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2d9ebc30-724f-4ab2-a492-6ee9974ac337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '429462c6a425456780bf3de30a7cedaa', 'request_id': 'req-2d9ebc30-724f-4ab2-a492-6ee9974ac337'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:42.411909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2d9ebc30-724f-4ab2-a492-6ee9974ac337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '429462c6a425456780bf3de30a7cedaa', 'request_id': 'req-2d9ebc30-724f-4ab2-a492-6ee9974ac337'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:42.419071 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2d9ebc30-724f-4ab2-a492-6ee9974ac337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '429462c6a425456780bf3de30a7cedaa', 'request_id': 'req-2d9ebc30-724f-4ab2-a492-6ee9974ac337'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:42.429691 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2d9ebc30-724f-4ab2-a492-6ee9974ac337 admin admin] The token cache is being invalidate because project 429462c6a425456780bf3de30a7cedaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:58:42.430006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2d9ebc30-724f-4ab2-a492-6ee9974ac337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:42.430645 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2d9ebc30-724f-4ab2-a492-6ee9974ac337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:42.432142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1126/2211] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:58:42 2022] DELETE /identity/v3/projects/429462c6a425456780bf3de30a7cedaa => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:58:42.437227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d34583-b70e-4705-9264-9d1d0d9c2c8b None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:42.437692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d34583-b70e-4705-9264-9d1d0d9c2c8b None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:42.437913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d34583-b70e-4705-9264-9d1d0d9c2c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:42.459964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-26d34583-b70e-4705-9264-9d1d0d9c2c8b None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:58:42.461296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-26d34583-b70e-4705-9264-9d1d0d9c2c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:42.480284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:58:42.521417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1086/2212] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:58:42.529557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547758d-6e13-4520-ab1c-428b3eb3e460 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:42.529864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547758d-6e13-4520-ab1c-428b3eb3e460 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:42.530143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547758d-6e13-4520-ab1c-428b3eb3e460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:42.550002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e547758d-6e13-4520-ab1c-428b3eb3e460 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:58:42.551211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e547758d-6e13-4520-ab1c-428b3eb3e460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:42.559123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1127/2213] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:58:42.563557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d34583-b70e-4705-9264-9d1d0d9c2c8b None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:42.574835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d34583-b70e-4705-9264-9d1d0d9c2c8b None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:42.579703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64ebd2-8bd6-4f6a-bea3-b5da9b3fddd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:42.580298 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64ebd2-8bd6-4f6a-bea3-b5da9b3fddd5 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:42.580576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64ebd2-8bd6-4f6a-bea3-b5da9b3fddd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:42.580848 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64ebd2-8bd6-4f6a-bea3-b5da9b3fddd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:42.581482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64ebd2-8bd6-4f6a-bea3-b5da9b3fddd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 19:58:42.581781 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64ebd2-8bd6-4f6a-bea3-b5da9b3fddd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:42.584088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64ebd2-8bd6-4f6a-bea3-b5da9b3fddd5 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:42.596267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1087/2214] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 19:58:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 19:58:42.600464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547758d-6e13-4520-ab1c-428b3eb3e460 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:42.609907 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547758d-6e13-4520-ab1c-428b3eb3e460 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:42.614548 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14291efc-c001-4d6c-b0c9-613efc46f97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:42.615140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14291efc-c001-4d6c-b0c9-613efc46f97e admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:42.615415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14291efc-c001-4d6c-b0c9-613efc46f97e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:42.615680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14291efc-c001-4d6c-b0c9-613efc46f97e admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:42.616348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14291efc-c001-4d6c-b0c9-613efc46f97e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:42.618276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14291efc-c001-4d6c-b0c9-613efc46f97e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:42.836091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1128/2215] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 19:58:42 2022] POST /identity/v3/projects => generated 375 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 19:58:42.840334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64ebd2-8bd6-4f6a-bea3-b5da9b3fddd5 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:42.850224 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64ebd2-8bd6-4f6a-bea3-b5da9b3fddd5 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:42.906884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939392e-613e-4d13-8bbf-fe6a73cf2ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:42.907590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939392e-613e-4d13-8bbf-fe6a73cf2ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:42.907903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939392e-613e-4d13-8bbf-fe6a73cf2ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:42.908161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939392e-613e-4d13-8bbf-fe6a73cf2ea2 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:42.908801 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939392e-613e-4d13-8bbf-fe6a73cf2ea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:42.910859 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d939392e-613e-4d13-8bbf-fe6a73cf2ea2 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:43.047455 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1088/2216] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 19:58:42 2022] POST /identity/v3/users => generated 288 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 19:58:43.054057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14291efc-c001-4d6c-b0c9-613efc46f97e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:43.089399 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14291efc-c001-4d6c-b0c9-613efc46f97e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:43.122769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1df43f-0f0b-4e8e-b471-364ad86706e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:43.126471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1df43f-0f0b-4e8e-b471-364ad86706e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:43.126471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1df43f-0f0b-4e8e-b471-364ad86706e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:43.126471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1df43f-0f0b-4e8e-b471-364ad86706e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:43.126471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1df43f-0f0b-4e8e-b471-364ad86706e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:43.126471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1df43f-0f0b-4e8e-b471-364ad86706e0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:43.138842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1129/2217] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:58:43.143094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939392e-613e-4d13-8bbf-fe6a73cf2ea2 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:43.154766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939392e-613e-4d13-8bbf-fe6a73cf2ea2 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:43.161507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464bfcb-1b19-4a4b-a762-480b49c0c103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:43.162160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bfcb-1b19-4a4b-a762-480b49c0c103 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:43.162442 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bfcb-1b19-4a4b-a762-480b49c0c103 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:43.162718 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bfcb-1b19-4a4b-a762-480b49c0c103 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:43.163431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464bfcb-1b19-4a4b-a762-480b49c0c103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:43.165051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464bfcb-1b19-4a4b-a762-480b49c0c103 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:43.180704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1089/2218] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 19:58:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 19:58:43.184007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1df43f-0f0b-4e8e-b471-364ad86706e0 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:43.196548 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1df43f-0f0b-4e8e-b471-364ad86706e0 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:43.202450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4021b1-5e0b-4e04-b6de-7661a1d61701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:43.204493 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4021b1-5e0b-4e04-b6de-7661a1d61701 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:43.218857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4021b1-5e0b-4e04-b6de-7661a1d61701 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:43.218857 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4021b1-5e0b-4e04-b6de-7661a1d61701 admin admin] PATH_INFO: `/v3/projects/352679873aec46a38e6eef234a4f5e23/users/ec96050b3f4b4fbe8ad7b39d64cea491/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:43.219360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464bfcb-1b19-4a4b-a762-480b49c0c103 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:43.223664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c464bfcb-1b19-4a4b-a762-480b49c0c103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:43.242257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4021b1-5e0b-4e04-b6de-7661a1d61701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=352679873aec46a38e6eef234a4f5e23, user_id=ec96050b3f4b4fbe8ad7b39d64cea491, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:43.245212 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4021b1-5e0b-4e04-b6de-7661a1d61701 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:43.326928 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464bfcb-1b19-4a4b-a762-480b49c0c103 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:43.334397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1130/2219] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 19:58:43 2022] PUT /identity/v3/projects/352679873aec46a38e6eef234a4f5e23/users/ec96050b3f4b4fbe8ad7b39d64cea491/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:58:43.346802 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ffbb5-04c6-4101-85b4-f0880841aa36 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:43.347294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ffbb5-04c6-4101-85b4-f0880841aa36 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:43.348380 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595ffbb5-04c6-4101-85b4-f0880841aa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:43.368483 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:58:43.386232 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-595ffbb5-04c6-4101-85b4-f0880841aa36 None None] MFA Rules not processed for user `ec96050b3f4b4fbe8ad7b39d64cea491`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:58:43.387890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-595ffbb5-04c6-4101-85b4-f0880841aa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:43.402958 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:58:43.423105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a2b53-972d-44d2-b78c-ae3d721bc494 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03bc280d0395479981b7c7b08c96001b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:43.423719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a2b53-972d-44d2-b78c-ae3d721bc494 service placement] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:43.423992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a2b53-972d-44d2-b78c-ae3d721bc494 service placement] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:43.424251 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a2b53-972d-44d2-b78c-ae3d721bc494 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:43.426926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a12a2b53-972d-44d2-b78c-ae3d721bc494 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:43.440677 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a2b53-972d-44d2-b78c-ae3d721bc494 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:43.442410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a2b53-972d-44d2-b78c-ae3d721bc494 service placement] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:43.454279 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1090/2220] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:58:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 19:58:43.492537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1131/2221] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:58:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 19:58:43.505263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a2b53-972d-44d2-b78c-ae3d721bc494 service placement] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:43.506923 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a12a2b53-972d-44d2-b78c-ae3d721bc494 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:43.582689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a2b53-972d-44d2-b78c-ae3d721bc494 service placement] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:43.586424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5d6ac-6711-4b54-abd9-d6e37a2e4059 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:43.587060 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5d6ac-6711-4b54-abd9-d6e37a2e4059 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:43.587300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5d6ac-6711-4b54-abd9-d6e37a2e4059 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:43.587570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5d6ac-6711-4b54-abd9-d6e37a2e4059 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:43.597125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5d6ac-6711-4b54-abd9-d6e37a2e4059 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:43.599327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5d6ac-6711-4b54-abd9-d6e37a2e4059 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:43.608630 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1091/2222] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:58:43 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:58:49.175050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e21da-9692-4cbc-9df1-9580b3a3f13f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:49.175586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e21da-9692-4cbc-9df1-9580b3a3f13f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:49.175586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e21da-9692-4cbc-9df1-9580b3a3f13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:49.199137 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-067e21da-9692-4cbc-9df1-9580b3a3f13f None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:58:49.201568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-067e21da-9692-4cbc-9df1-9580b3a3f13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:49.263266 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1132/2223] 10.222.0.42 () {58 vars in 967 bytes} [Wed Sep 14 19:58:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:58:49.278804 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5d6ac-6711-4b54-abd9-d6e37a2e4059 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:49.280358 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c5b5d6ac-6711-4b54-abd9-d6e37a2e4059 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:49.368394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5d6ac-6711-4b54-abd9-d6e37a2e4059 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:49.373667 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de5f16-2752-4317-afa4-327f93f0cc3e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:49.374308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de5f16-2752-4317-afa4-327f93f0cc3e service swift] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:49.374587 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de5f16-2752-4317-afa4-327f93f0cc3e service swift] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:49.374872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de5f16-2752-4317-afa4-327f93f0cc3e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:49.383350 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de5f16-2752-4317-afa4-327f93f0cc3e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:49.384947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de5f16-2752-4317-afa4-327f93f0cc3e service swift] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:49.403260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1092/2224] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 19:58:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 19:58:55.472421 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e21da-9692-4cbc-9df1-9580b3a3f13f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:55.543016 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e21da-9692-4cbc-9df1-9580b3a3f13f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:55.547501 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d7177-9a60-4da2-a6e4-edfb2f5d3d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:55.548150 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d7177-9a60-4da2-a6e4-edfb2f5d3d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:55.548424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d7177-9a60-4da2-a6e4-edfb2f5d3d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:55.548688 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4d7177-9a60-4da2-a6e4-edfb2f5d3d31 admin admin] PATH_INFO: `/v3/users/ec96050b3f4b4fbe8ad7b39d64cea491` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:55.549823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d7177-9a60-4da2-a6e4-edfb2f5d3d31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec96050b3f4b4fbe8ad7b39d64cea491)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:55.551779 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4d7177-9a60-4da2-a6e4-edfb2f5d3d31 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:55.745649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1d4d7177-9a60-4da2-a6e4-edfb2f5d3d31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec96050b3f4b4fbe8ad7b39d64cea491'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:55.846428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1d4d7177-9a60-4da2-a6e4-edfb2f5d3d31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec96050b3f4b4fbe8ad7b39d64cea491'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:55.849284 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:58:55.860572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1d4d7177-9a60-4da2-a6e4-edfb2f5d3d31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec96050b3f4b4fbe8ad7b39d64cea491'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:55.861298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1d4d7177-9a60-4da2-a6e4-edfb2f5d3d31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec96050b3f4b4fbe8ad7b39d64cea491'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:55.861926 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1d4d7177-9a60-4da2-a6e4-edfb2f5d3d31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec96050b3f4b4fbe8ad7b39d64cea491'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:55.874003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1133/2225] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 19:58:55 2022] DELETE /identity/v3/users/ec96050b3f4b4fbe8ad7b39d64cea491 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:58:55.878586 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06501605-56be-4109-88e4-97776fd252da None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:55.879521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06501605-56be-4109-88e4-97776fd252da None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:55.879644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06501605-56be-4109-88e4-97776fd252da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:55.900254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-06501605-56be-4109-88e4-97776fd252da None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:58:55.901441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-06501605-56be-4109-88e4-97776fd252da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:55.946400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:58:55.958901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1093/2226] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:58:55.968546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d7177-9a60-4da2-a6e4-edfb2f5d3d31 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:55.970018 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1d4d7177-9a60-4da2-a6e4-edfb2f5d3d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:56.025392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4d7177-9a60-4da2-a6e4-edfb2f5d3d31 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:56.028940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b2d4e-7432-4656-b705-dcc3df35b8fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:56.029535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b2d4e-7432-4656-b705-dcc3df35b8fa service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:56.029820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b2d4e-7432-4656-b705-dcc3df35b8fa service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:56.030108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b2d4e-7432-4656-b705-dcc3df35b8fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:56.038911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b2d4e-7432-4656-b705-dcc3df35b8fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:56.040312 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b2d4e-7432-4656-b705-dcc3df35b8fa service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:56.049424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1134/2227] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 19:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 19:58:56.507272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06501605-56be-4109-88e4-97776fd252da None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 19:58:56.508684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-06501605-56be-4109-88e4-97776fd252da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:58:56.520807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06501605-56be-4109-88e4-97776fd252da None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 19:58:56.525412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7613f0c-b519-43d4-858b-3d8474a86ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 19:58:56.526123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7613f0c-b519-43d4-858b-3d8474a86ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:58:56.526396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7613f0c-b519-43d4-858b-3d8474a86ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:58:56.526663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7613f0c-b519-43d4-858b-3d8474a86ecc admin admin] PATH_INFO: `/v3/projects/352679873aec46a38e6eef234a4f5e23` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:58:56.527789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7613f0c-b519-43d4-858b-3d8474a86ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=352679873aec46a38e6eef234a4f5e23)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 19:58:56.529786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7613f0c-b519-43d4-858b-3d8474a86ecc admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 19:58:56.708701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-f7613f0c-b519-43d4-858b-3d8474a86ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '352679873aec46a38e6eef234a4f5e23', 'request_id': 'req-f7613f0c-b519-43d4-858b-3d8474a86ecc'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:56.714265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-f7613f0c-b519-43d4-858b-3d8474a86ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '352679873aec46a38e6eef234a4f5e23', 'request_id': 'req-f7613f0c-b519-43d4-858b-3d8474a86ecc'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:56.720516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-f7613f0c-b519-43d4-858b-3d8474a86ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '352679873aec46a38e6eef234a4f5e23', 'request_id': 'req-f7613f0c-b519-43d4-858b-3d8474a86ecc'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:56.822526 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-f7613f0c-b519-43d4-858b-3d8474a86ecc admin admin] The token cache is being invalidate because project 352679873aec46a38e6eef234a4f5e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 19:58:56.822814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-f7613f0c-b519-43d4-858b-3d8474a86ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:56.823437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-f7613f0c-b519-43d4-858b-3d8474a86ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 19:58:56.824906 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1094/2228] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 19:58:56 2022] DELETE /identity/v3/projects/352679873aec46a38e6eef234a4f5e23 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 19:59:06.102942 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=36,keystone:DELETE=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:59:06.819119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:59:14.809839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da262910-607e-4768-af02-02f1f97af40e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:59:14.810670 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da262910-607e-4768-af02-02f1f97af40e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:59:14.810942 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da262910-607e-4768-af02-02f1f97af40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:59:14.830655 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-da262910-607e-4768-af02-02f1f97af40e None None] MFA Rules not processed for user `be0920bd9b934f039e5e4de8039346b8`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:59:14.831852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-da262910-607e-4768-af02-02f1f97af40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:59:14.891300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1135/2229] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:59:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:59:24.884615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=25 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 19:59:39.981619 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ea2c1-314c-4276-aeda-94928e26aaa9 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 19:59:39.982163 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ea2c1-314c-4276-aeda-94928e26aaa9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 19:59:39.982272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ea2c1-314c-4276-aeda-94928e26aaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 19:59:40.000904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-399ea2c1-314c-4276-aeda-94928e26aaa9 None None] MFA Rules not processed for user `a66d0ec3a156469980551f9bc9034d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 19:59:40.002134 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-399ea2c1-314c-4276-aeda-94928e26aaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 19:59:40.054764 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1095/2230] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 19:59:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 19:59:50.048118 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=25 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:00:10.147468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304b2dc-5d6b-4a79-97c4-ce7530517279 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:10.147982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304b2dc-5d6b-4a79-97c4-ce7530517279 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:10.148099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304b2dc-5d6b-4a79-97c4-ce7530517279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:10.166826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6304b2dc-5d6b-4a79-97c4-ce7530517279 None None] MFA Rules not processed for user `2bf1c7a6cc2e4b1294e7b162a8aba738`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:00:10.168008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6304b2dc-5d6b-4a79-97c4-ce7530517279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:10.222431 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1136/2231] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:00:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:00:20.218544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=25 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:00:20.763814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4e2d0-d168-4db6-8826-63744ed6563d None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:20.764335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4e2d0-d168-4db6-8826-63744ed6563d None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:20.764335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4e2d0-d168-4db6-8826-63744ed6563d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:20.784151 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-77d4e2d0-d168-4db6-8826-63744ed6563d None None] MFA Rules not processed for user `766dd5a650fc4855980250969854f08d`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:00:20.785462 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-77d4e2d0-d168-4db6-8826-63744ed6563d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:20.842554 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1096/2232] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:00:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:00:22.462408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15192cbe-4a2c-412b-8db1-0c0f8d674fcf None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:22.462882 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15192cbe-4a2c-412b-8db1-0c0f8d674fcf None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:22.462976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15192cbe-4a2c-412b-8db1-0c0f8d674fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:22.482576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-15192cbe-4a2c-412b-8db1-0c0f8d674fcf None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:00:22.483780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-15192cbe-4a2c-412b-8db1-0c0f8d674fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:22.537498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1137/2233] 10.222.0.42 () {58 vars in 967 bytes} [Wed Sep 14 20:00:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:00:22.664160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4e2d0-d168-4db6-8826-63744ed6563d None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:22.666272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-77d4e2d0-d168-4db6-8826-63744ed6563d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:22.737318 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4e2d0-d168-4db6-8826-63744ed6563d None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:22.742219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1cdac-8e80-4614-8815-ba4879a97162 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:22.742861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1cdac-8e80-4614-8815-ba4879a97162 service swift] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:22.743136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1cdac-8e80-4614-8815-ba4879a97162 service swift] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:22.743406 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1cdac-8e80-4614-8815-ba4879a97162 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:22.751123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1cdac-8e80-4614-8815-ba4879a97162 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:22.752680 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b1cdac-8e80-4614-8815-ba4879a97162 service swift] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:22.760446 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1097/2234] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 20:00:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 20:00:28.476423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ce735-cba7-4342-b68e-45ba6cd16dfc None None] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:28.477025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ce735-cba7-4342-b68e-45ba6cd16dfc None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:28.477621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3ce735-cba7-4342-b68e-45ba6cd16dfc None None] PATH_INFO: `/` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:28.479081 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1138/2235] 10.222.0.42 () {54 vars in 921 bytes} [Wed Sep 14 20:00:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 20:00:28.506961 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf7463-9d30-4834-b683-b650021953e2 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:28.507553 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf7463-9d30-4834-b683-b650021953e2 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:28.507798 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf7463-9d30-4834-b683-b650021953e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:28.537152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-8aaf7463-9d30-4834-b683-b650021953e2 None None] MFA Rules not processed for user `03bc280d0395479981b7c7b08c96001b`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:00:28.538604 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8aaf7463-9d30-4834-b683-b650021953e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:28.647891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1098/2236] 10.222.0.42 () {58 vars in 1018 bytes} [Wed Sep 14 20:00:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:00:28.668363 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ce735-cba7-4342-b68e-45ba6cd16dfc None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:28.670392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0e3ce735-cba7-4342-b68e-45ba6cd16dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:28.687548 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3ce735-cba7-4342-b68e-45ba6cd16dfc None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:28.695328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e674e25-5a4a-4351-8e6c-185398ed5126 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03bc280d0395479981b7c7b08c96001b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:28.696104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e674e25-5a4a-4351-8e6c-185398ed5126 service placement] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:28.696426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e674e25-5a4a-4351-8e6c-185398ed5126 service placement] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:28.696697 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e674e25-5a4a-4351-8e6c-185398ed5126 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:28.709318 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e674e25-5a4a-4351-8e6c-185398ed5126 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:28.725887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e674e25-5a4a-4351-8e6c-185398ed5126 service placement] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:28.725887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1139/2237] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:00:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:00:34.234025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf7463-9d30-4834-b683-b650021953e2 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:34.235607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8aaf7463-9d30-4834-b683-b650021953e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:34.276981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:00:34.307098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf7463-9d30-4834-b683-b650021953e2 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:34.311632 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8489d-5e56-43a7-8b83-baab4e9fe0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:34.312284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8489d-5e56-43a7-8b83-baab4e9fe0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:34.312780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8489d-5e56-43a7-8b83-baab4e9fe0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:34.313081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8489d-5e56-43a7-8b83-baab4e9fe0f9 admin admin] PATH_INFO: `/v3/users/0ebafc08399c4853959800883742eb78` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:34.314313 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8489d-5e56-43a7-8b83-baab4e9fe0f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ebafc08399c4853959800883742eb78)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:34.316276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8489d-5e56-43a7-8b83-baab4e9fe0f9 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:34.485490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e7b8489d-5e56-43a7-8b83-baab4e9fe0f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ebafc08399c4853959800883742eb78'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:34.497854 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e7b8489d-5e56-43a7-8b83-baab4e9fe0f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ebafc08399c4853959800883742eb78'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:34.551356 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e7b8489d-5e56-43a7-8b83-baab4e9fe0f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ebafc08399c4853959800883742eb78'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:34.551980 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e7b8489d-5e56-43a7-8b83-baab4e9fe0f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ebafc08399c4853959800883742eb78'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:34.559705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e7b8489d-5e56-43a7-8b83-baab4e9fe0f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ebafc08399c4853959800883742eb78'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:34.565553 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1099/2238] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:00:34 2022] DELETE /identity/v3/users/0ebafc08399c4853959800883742eb78 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:00:34.814225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e674e25-5a4a-4351-8e6c-185398ed5126 service placement] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:34.842866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e674e25-5a4a-4351-8e6c-185398ed5126 service placement] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:34.920242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:34.920888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:34.921395 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:34.921586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] PATH_INFO: `/v3/projects/11fae649642b4703b649d02268bdb4eb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:34.922901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11fae649642b4703b649d02268bdb4eb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:34.925466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:35.031667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11fae649642b4703b649d02268bdb4eb', 'request_id': 'req-dd38794c-a375-4668-a234-dd249a88de61'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:35.111184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11fae649642b4703b649d02268bdb4eb', 'request_id': 'req-dd38794c-a375-4668-a234-dd249a88de61'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:35.119458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11fae649642b4703b649d02268bdb4eb', 'request_id': 'req-dd38794c-a375-4668-a234-dd249a88de61'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:35.131111 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] The token cache is being invalidate because project 11fae649642b4703b649d02268bdb4eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:00:35.131404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:35.132085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:35.133633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1140/2239] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:00:34 2022] DELETE /identity/v3/projects/11fae649642b4703b649d02268bdb4eb => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:00:35.137888 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8489d-5e56-43a7-8b83-baab4e9fe0f9 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:35.139316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e7b8489d-5e56-43a7-8b83-baab4e9fe0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:35.152969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8489d-5e56-43a7-8b83-baab4e9fe0f9 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:35.219791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559bb7c-7c97-4c9d-bd4a-75ffb0d5ad70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:35.220435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559bb7c-7c97-4c9d-bd4a-75ffb0d5ad70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:35.220709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559bb7c-7c97-4c9d-bd4a-75ffb0d5ad70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:35.220975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559bb7c-7c97-4c9d-bd4a-75ffb0d5ad70 admin admin] PATH_INFO: `/v3/users/5ce3c93ac7d04f6ea76fd82a2bc810cb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:35.222171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559bb7c-7c97-4c9d-bd4a-75ffb0d5ad70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce3c93ac7d04f6ea76fd82a2bc810cb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:35.224168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559bb7c-7c97-4c9d-bd4a-75ffb0d5ad70 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:35.446462 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0559bb7c-7c97-4c9d-bd4a-75ffb0d5ad70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce3c93ac7d04f6ea76fd82a2bc810cb'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:35.452307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:00:35.461789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0559bb7c-7c97-4c9d-bd4a-75ffb0d5ad70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce3c93ac7d04f6ea76fd82a2bc810cb'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:35.574518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0559bb7c-7c97-4c9d-bd4a-75ffb0d5ad70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce3c93ac7d04f6ea76fd82a2bc810cb'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:35.575156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0559bb7c-7c97-4c9d-bd4a-75ffb0d5ad70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce3c93ac7d04f6ea76fd82a2bc810cb'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:35.582859 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-0559bb7c-7c97-4c9d-bd4a-75ffb0d5ad70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce3c93ac7d04f6ea76fd82a2bc810cb'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:35.589475 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1100/2240] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:00:35 2022] DELETE /identity/v3/users/5ce3c93ac7d04f6ea76fd82a2bc810cb => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:00:35.878251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:35.879767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:35.893480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd38794c-a375-4668-a234-dd249a88de61 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:35.915723 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:00:35.953617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65ea26c-2f47-4e16-aaec-1f31f06c1892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:35.954276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ea26c-2f47-4e16-aaec-1f31f06c1892 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:35.954555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ea26c-2f47-4e16-aaec-1f31f06c1892 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:35.954825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65ea26c-2f47-4e16-aaec-1f31f06c1892 admin admin] PATH_INFO: `/v3/projects/64fbaaef29364174a40d699e0b9458af` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:35.956181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ea26c-2f47-4e16-aaec-1f31f06c1892 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64fbaaef29364174a40d699e0b9458af)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:35.958128 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65ea26c-2f47-4e16-aaec-1f31f06c1892 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:36.060792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e65ea26c-2f47-4e16-aaec-1f31f06c1892 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64fbaaef29364174a40d699e0b9458af', 'request_id': 'req-e65ea26c-2f47-4e16-aaec-1f31f06c1892'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:36.134959 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e65ea26c-2f47-4e16-aaec-1f31f06c1892 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64fbaaef29364174a40d699e0b9458af', 'request_id': 'req-e65ea26c-2f47-4e16-aaec-1f31f06c1892'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:36.141925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e65ea26c-2f47-4e16-aaec-1f31f06c1892 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64fbaaef29364174a40d699e0b9458af', 'request_id': 'req-e65ea26c-2f47-4e16-aaec-1f31f06c1892'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:36.152169 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e65ea26c-2f47-4e16-aaec-1f31f06c1892 admin admin] The token cache is being invalidate because project 64fbaaef29364174a40d699e0b9458af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:00:36.152449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e65ea26c-2f47-4e16-aaec-1f31f06c1892 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:36.153070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e65ea26c-2f47-4e16-aaec-1f31f06c1892 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:36.154632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1141/2241] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:00:35 2022] DELETE /identity/v3/projects/64fbaaef29364174a40d699e0b9458af => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:00:36.159866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b567f-6eea-46ee-a507-c0e11f3faf3b None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:36.160153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b567f-6eea-46ee-a507-c0e11f3faf3b None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:36.160437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b567f-6eea-46ee-a507-c0e11f3faf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:36.179649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-080b567f-6eea-46ee-a507-c0e11f3faf3b None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:00:36.181338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-080b567f-6eea-46ee-a507-c0e11f3faf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:36.242371 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1101/2242] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:00:36.250477 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f85eca-d97a-496c-a1b8-963063d61c7f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:36.250781 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f85eca-d97a-496c-a1b8-963063d61c7f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:36.251047 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f85eca-d97a-496c-a1b8-963063d61c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:36.271401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b5f85eca-d97a-496c-a1b8-963063d61c7f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:00:36.272718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b5f85eca-d97a-496c-a1b8-963063d61c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:36.280836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1142/2243] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:00:36.285292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b567f-6eea-46ee-a507-c0e11f3faf3b None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:36.295166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b567f-6eea-46ee-a507-c0e11f3faf3b None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:36.299804 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4049a6-33e1-4513-bc03-98bc19d1055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:36.300398 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4049a6-33e1-4513-bc03-98bc19d1055e admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:36.300677 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4049a6-33e1-4513-bc03-98bc19d1055e admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:36.300951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4049a6-33e1-4513-bc03-98bc19d1055e admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:36.301594 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4049a6-33e1-4513-bc03-98bc19d1055e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:00:36.301932 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4049a6-33e1-4513-bc03-98bc19d1055e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:36.303534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4049a6-33e1-4513-bc03-98bc19d1055e admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:36.315656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1102/2244] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:00:36.319750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f85eca-d97a-496c-a1b8-963063d61c7f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:36.330448 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f85eca-d97a-496c-a1b8-963063d61c7f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:36.335850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d2d31-a9df-48e7-bbad-fac3382999c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:36.336654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d2d31-a9df-48e7-bbad-fac3382999c8 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:36.337096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d2d31-a9df-48e7-bbad-fac3382999c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:36.337374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d2d31-a9df-48e7-bbad-fac3382999c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:36.338029 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d2d31-a9df-48e7-bbad-fac3382999c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:36.339937 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d2d31-a9df-48e7-bbad-fac3382999c8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:36.417154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1143/2245] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:00:36 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:00:36.421283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4049a6-33e1-4513-bc03-98bc19d1055e admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:36.432227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4049a6-33e1-4513-bc03-98bc19d1055e admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:36.494373 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f637ae7-38ea-45a5-aa69-2c50827a7518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:36.495088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f637ae7-38ea-45a5-aa69-2c50827a7518 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:36.495244 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f637ae7-38ea-45a5-aa69-2c50827a7518 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:36.495539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f637ae7-38ea-45a5-aa69-2c50827a7518 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:36.496160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f637ae7-38ea-45a5-aa69-2c50827a7518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:36.498153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f637ae7-38ea-45a5-aa69-2c50827a7518 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:36.603883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1103/2246] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:00:36 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:00:36.607915 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d2d31-a9df-48e7-bbad-fac3382999c8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:36.619503 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d2d31-a9df-48e7-bbad-fac3382999c8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:36.624068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9586f0f-30d6-44fc-b9dd-57a1d474755d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:36.624736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9586f0f-30d6-44fc-b9dd-57a1d474755d admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:36.625012 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9586f0f-30d6-44fc-b9dd-57a1d474755d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:36.625304 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9586f0f-30d6-44fc-b9dd-57a1d474755d admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:36.626076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9586f0f-30d6-44fc-b9dd-57a1d474755d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:36.627704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9586f0f-30d6-44fc-b9dd-57a1d474755d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:36.637418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1144/2247] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:00:36.641233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f637ae7-38ea-45a5-aa69-2c50827a7518 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:36.651362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f637ae7-38ea-45a5-aa69-2c50827a7518 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:36.655957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c383d-1434-46f7-8a71-74f6718ee04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:36.656541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c383d-1434-46f7-8a71-74f6718ee04a admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:36.656845 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c383d-1434-46f7-8a71-74f6718ee04a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:36.657119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c383d-1434-46f7-8a71-74f6718ee04a admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:36.657774 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c383d-1434-46f7-8a71-74f6718ee04a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:36.659412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c383d-1434-46f7-8a71-74f6718ee04a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:36.670361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1104/2248] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:00:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:00:36.674350 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9586f0f-30d6-44fc-b9dd-57a1d474755d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:36.684639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9586f0f-30d6-44fc-b9dd-57a1d474755d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:36.689170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd0632-9120-4770-9b81-2ed79a748e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:36.689818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd0632-9120-4770-9b81-2ed79a748e40 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:36.690099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd0632-9120-4770-9b81-2ed79a748e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:36.690367 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd0632-9120-4770-9b81-2ed79a748e40 admin admin] PATH_INFO: `/v3/projects/2d5c2273800e43a096bce8807c31fbf5/users/04af0db8835d427092c076d35b3e67d9/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:36.704311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd0632-9120-4770-9b81-2ed79a748e40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d5c2273800e43a096bce8807c31fbf5, user_id=04af0db8835d427092c076d35b3e67d9, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:36.707243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd0632-9120-4770-9b81-2ed79a748e40 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:36.799183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1145/2249] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:00:36 2022] PUT /identity/v3/projects/2d5c2273800e43a096bce8807c31fbf5/users/04af0db8835d427092c076d35b3e67d9/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:00:36.805285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b746c8b-a3d8-48c7-80e5-57f993a73639 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:36.805285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b746c8b-a3d8-48c7-80e5-57f993a73639 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:36.805285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b746c8b-a3d8-48c7-80e5-57f993a73639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:36.823837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-3b746c8b-a3d8-48c7-80e5-57f993a73639 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:00:36.825094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-3b746c8b-a3d8-48c7-80e5-57f993a73639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:36.856219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:00:36.890652 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1105/2250] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:00:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:00:36.900765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd0632-9120-4770-9b81-2ed79a748e40 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:36.902632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-edbd0632-9120-4770-9b81-2ed79a748e40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:36.954940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:00:36.958977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd0632-9120-4770-9b81-2ed79a748e40 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:36.963159 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a96dc-e979-40d1-83d7-9edd317ebaad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:36.963757 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a96dc-e979-40d1-83d7-9edd317ebaad service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:36.964054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a96dc-e979-40d1-83d7-9edd317ebaad service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:36.964321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a96dc-e979-40d1-83d7-9edd317ebaad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:36.973031 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a96dc-e979-40d1-83d7-9edd317ebaad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:36.974533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a96dc-e979-40d1-83d7-9edd317ebaad service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:36.984423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1146/2251] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:00:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:00:38.873401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b746c8b-a3d8-48c7-80e5-57f993a73639 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:38.874901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-3b746c8b-a3d8-48c7-80e5-57f993a73639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:38.888258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b746c8b-a3d8-48c7-80e5-57f993a73639 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:38.891957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabcea8-4a55-4cec-b7f9-471c445f1521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:38.892860 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabcea8-4a55-4cec-b7f9-471c445f1521 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:38.893139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabcea8-4a55-4cec-b7f9-471c445f1521 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:38.893429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabcea8-4a55-4cec-b7f9-471c445f1521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:38.895238 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-edabcea8-4a55-4cec-b7f9-471c445f1521 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:38.970672 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabcea8-4a55-4cec-b7f9-471c445f1521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:38.972260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabcea8-4a55-4cec-b7f9-471c445f1521 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:38.983073 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1106/2252] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:00:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:00:44.191451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f995b8-c41e-49fc-b26b-316e7a55aa7c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:44.191909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f995b8-c41e-49fc-b26b-316e7a55aa7c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:44.192278 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f995b8-c41e-49fc-b26b-316e7a55aa7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:44.228472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a8f995b8-c41e-49fc-b26b-316e7a55aa7c None None] MFA Rules not processed for user `04af0db8835d427092c076d35b3e67d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:00:44.229785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a8f995b8-c41e-49fc-b26b-316e7a55aa7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:44.292587 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabcea8-4a55-4cec-b7f9-471c445f1521 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:44.302693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabcea8-4a55-4cec-b7f9-471c445f1521 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:44.306816 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046b53b-1afc-49c8-a887-8e3a1a52a08d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:44.307505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046b53b-1afc-49c8-a887-8e3a1a52a08d service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:44.308001 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046b53b-1afc-49c8-a887-8e3a1a52a08d service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:44.308492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b046b53b-1afc-49c8-a887-8e3a1a52a08d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:44.310553 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b046b53b-1afc-49c8-a887-8e3a1a52a08d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:44.311556 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1147/2253] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:00:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:00:44.315870 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f995b8-c41e-49fc-b26b-316e7a55aa7c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:44.325777 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046b53b-1afc-49c8-a887-8e3a1a52a08d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:44.327489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b046b53b-1afc-49c8-a887-8e3a1a52a08d service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:44.328178 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f995b8-c41e-49fc-b26b-316e7a55aa7c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:44.333558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f40698-870b-4e4a-93a9-3f8d19696151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:44.334343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f40698-870b-4e4a-93a9-3f8d19696151 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:44.334721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f40698-870b-4e4a-93a9-3f8d19696151 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:44.335111 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f40698-870b-4e4a-93a9-3f8d19696151 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:44.335868 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f40698-870b-4e4a-93a9-3f8d19696151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:44.338443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f40698-870b-4e4a-93a9-3f8d19696151 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:44.340071 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1107/2254] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:00:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:00:44.450971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1148/2255] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:00:44 2022] POST /identity/v3/projects => generated 373 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:00:44.455737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046b53b-1afc-49c8-a887-8e3a1a52a08d service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:44.467805 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b046b53b-1afc-49c8-a887-8e3a1a52a08d service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:44.554685 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583565f6-9517-426c-9047-d11dc4ccd6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:44.555332 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583565f6-9517-426c-9047-d11dc4ccd6ce admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:44.555607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583565f6-9517-426c-9047-d11dc4ccd6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:44.555876 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583565f6-9517-426c-9047-d11dc4ccd6ce admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:44.556505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583565f6-9517-426c-9047-d11dc4ccd6ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:44.558476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583565f6-9517-426c-9047-d11dc4ccd6ce admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:44.655799 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1108/2256] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:00:44 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:00:44.659953 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f40698-870b-4e4a-93a9-3f8d19696151 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:44.670600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f40698-870b-4e4a-93a9-3f8d19696151 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:44.675246 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d5aa5-610d-435a-8fbb-ad4455c2b078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:44.675848 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d5aa5-610d-435a-8fbb-ad4455c2b078 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:44.676120 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d5aa5-610d-435a-8fbb-ad4455c2b078 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:44.676387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d5aa5-610d-435a-8fbb-ad4455c2b078 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:44.677013 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d5aa5-610d-435a-8fbb-ad4455c2b078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:44.678643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d5aa5-610d-435a-8fbb-ad4455c2b078 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:44.691758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1149/2257] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:00:44.696621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583565f6-9517-426c-9047-d11dc4ccd6ce admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:44.707262 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583565f6-9517-426c-9047-d11dc4ccd6ce admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:44.712911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ac73a-1361-43b7-aed4-e4fee9140c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:44.713577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac73a-1361-43b7-aed4-e4fee9140c0e admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:44.713883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac73a-1361-43b7-aed4-e4fee9140c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:44.714152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac73a-1361-43b7-aed4-e4fee9140c0e admin admin] PATH_INFO: `/v3/projects/363d06d74a254e9ca1f390ca4c961096/users/e024f51143c64b8eb2849762bad968ea/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:44.729674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ac73a-1361-43b7-aed4-e4fee9140c0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=363d06d74a254e9ca1f390ca4c961096, user_id=e024f51143c64b8eb2849762bad968ea, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:44.732349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4ac73a-1361-43b7-aed4-e4fee9140c0e admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:44.786951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1109/2258] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:00:44 2022] PUT /identity/v3/projects/363d06d74a254e9ca1f390ca4c961096/users/e024f51143c64b8eb2849762bad968ea/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:00:44.790986 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d5aa5-610d-435a-8fbb-ad4455c2b078 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:44.801208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d5aa5-610d-435a-8fbb-ad4455c2b078 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:44.886103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feaf93b-854d-4e44-8cde-a637af025e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:44.886760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaf93b-854d-4e44-8cde-a637af025e35 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:44.887060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaf93b-854d-4e44-8cde-a637af025e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:44.887328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feaf93b-854d-4e44-8cde-a637af025e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:44.887967 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feaf93b-854d-4e44-8cde-a637af025e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:44.889607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feaf93b-854d-4e44-8cde-a637af025e35 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:44.901838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1150/2259] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:00:44.906919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ac73a-1361-43b7-aed4-e4fee9140c0e admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:44.918025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ac73a-1361-43b7-aed4-e4fee9140c0e admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:44.922771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc5f4f1-8362-4741-a3ea-4b815c8067ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:44.923591 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5f4f1-8362-4741-a3ea-4b815c8067ed admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:44.923864 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5f4f1-8362-4741-a3ea-4b815c8067ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:44.924131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc5f4f1-8362-4741-a3ea-4b815c8067ed admin admin] PATH_INFO: `/v3/projects/363d06d74a254e9ca1f390ca4c961096/users/e024f51143c64b8eb2849762bad968ea/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:44.926154 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5f4f1-8362-4741-a3ea-4b815c8067ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=363d06d74a254e9ca1f390ca4c961096, user_id=e024f51143c64b8eb2849762bad968ea, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:44.928963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc5f4f1-8362-4741-a3ea-4b815c8067ed admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:44.938177 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:00:44.944252 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1110/2260] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:00:44 2022] PUT /identity/v3/projects/363d06d74a254e9ca1f390ca4c961096/users/e024f51143c64b8eb2849762bad968ea/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:00:51.584218 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04f74a-2525-4747-b9f2-b352500497c8 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:51.584733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04f74a-2525-4747-b9f2-b352500497c8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:51.585052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04f74a-2525-4747-b9f2-b352500497c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:51.617205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1e04f74a-2525-4747-b9f2-b352500497c8 None None] MFA Rules not processed for user `e024f51143c64b8eb2849762bad968ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:00:51.618509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1e04f74a-2525-4747-b9f2-b352500497c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:51.654776 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:00:51.690855 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67211b95-8737-44cd-adc5-e98fb4207d05 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:51.691726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67211b95-8737-44cd-adc5-e98fb4207d05 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:51.692146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67211b95-8737-44cd-adc5-e98fb4207d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:51.709710 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1151/2261] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:00:51 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:00:51.714864 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-67211b95-8737-44cd-adc5-e98fb4207d05 None None] MFA Rules not processed for user `be0920bd9b934f039e5e4de8039346b8`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:00:51.716204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-67211b95-8737-44cd-adc5-e98fb4207d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:51.716603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04f74a-2525-4747-b9f2-b352500497c8 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:51.718005 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1e04f74a-2525-4747-b9f2-b352500497c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:51.782973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1111/2262] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:00:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:00:51.801077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04f74a-2525-4747-b9f2-b352500497c8 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:51.808300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d8964-c9a4-4eb5-ad69-f24e2b7dd525 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:51.808977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d8964-c9a4-4eb5-ad69-f24e2b7dd525 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:51.809432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d8964-c9a4-4eb5-ad69-f24e2b7dd525 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:51.809761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d8964-c9a4-4eb5-ad69-f24e2b7dd525 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:51.872863 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d8964-c9a4-4eb5-ad69-f24e2b7dd525 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:51.874785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12d8964-c9a4-4eb5-ad69-f24e2b7dd525 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:51.884799 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1152/2263] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:00:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:00:52.107535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67211b95-8737-44cd-adc5-e98fb4207d05 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:52.161962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67211b95-8737-44cd-adc5-e98fb4207d05 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:52.166250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3108bb-ad01-493f-8a6c-f6626c659516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:52.166851 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3108bb-ad01-493f-8a6c-f6626c659516 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:52.167170 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3108bb-ad01-493f-8a6c-f6626c659516 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:52.167459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3108bb-ad01-493f-8a6c-f6626c659516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:52.169255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0b3108bb-ad01-493f-8a6c-f6626c659516 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:52.181737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3108bb-ad01-493f-8a6c-f6626c659516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:52.183215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3108bb-ad01-493f-8a6c-f6626c659516 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:52.193754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1112/2264] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:00:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:00:52.683095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d8964-c9a4-4eb5-ad69-f24e2b7dd525 service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:52.684698 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c12d8964-c9a4-4eb5-ad69-f24e2b7dd525 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:52.781468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12d8964-c9a4-4eb5-ad69-f24e2b7dd525 service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:52.788767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3b9a7-e6d7-4fab-aa56-35a2661e6573 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03bc280d0395479981b7c7b08c96001b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:52.791464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3b9a7-e6d7-4fab-aa56-35a2661e6573 service placement] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:52.791464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3b9a7-e6d7-4fab-aa56-35a2661e6573 service placement] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:52.791464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3b9a7-e6d7-4fab-aa56-35a2661e6573 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:52.793568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b8c3b9a7-e6d7-4fab-aa56-35a2661e6573 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:52.808532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:00:52.812772 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3b9a7-e6d7-4fab-aa56-35a2661e6573 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:52.815985 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3b9a7-e6d7-4fab-aa56-35a2661e6573 service placement] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:52.849682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1153/2265] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:00:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:00:54.865184 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91da966-fcfe-4f66-b685-1b9e478daa54 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:54.865608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91da966-fcfe-4f66-b685-1b9e478daa54 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:54.865731 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91da966-fcfe-4f66-b685-1b9e478daa54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:54.887393 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-a91da966-fcfe-4f66-b685-1b9e478daa54 None None] MFA Rules not processed for user `04af0db8835d427092c076d35b3e67d9`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:00:54.888639 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a91da966-fcfe-4f66-b685-1b9e478daa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:54.913921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1113/2266] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:00:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:00:59.619261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3b9a7-e6d7-4fab-aa56-35a2661e6573 service placement] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:00:59.620764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b8c3b9a7-e6d7-4fab-aa56-35a2661e6573 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:00:59.692516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3b9a7-e6d7-4fab-aa56-35a2661e6573 service placement] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:00:59.698263 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2e944-4871-417e-8574-c49fa60b46b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:00:59.698961 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2e944-4871-417e-8574-c49fa60b46b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:00:59.699339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2e944-4871-417e-8574-c49fa60b46b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:00:59.699568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f2e944-4871-417e-8574-c49fa60b46b0 admin admin] PATH_INFO: `/v3/users/a66d0ec3a156469980551f9bc9034d4e` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:00:59.701136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2e944-4871-417e-8574-c49fa60b46b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a66d0ec3a156469980551f9bc9034d4e)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:00:59.703152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f2e944-4871-417e-8574-c49fa60b46b0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:00:59.892025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-12f2e944-4871-417e-8574-c49fa60b46b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a66d0ec3a156469980551f9bc9034d4e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:59.969233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-12f2e944-4871-417e-8574-c49fa60b46b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a66d0ec3a156469980551f9bc9034d4e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:59.982043 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-12f2e944-4871-417e-8574-c49fa60b46b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a66d0ec3a156469980551f9bc9034d4e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:59.982605 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-12f2e944-4871-417e-8574-c49fa60b46b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a66d0ec3a156469980551f9bc9034d4e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:59.983279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-12f2e944-4871-417e-8574-c49fa60b46b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a66d0ec3a156469980551f9bc9034d4e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:00:59.996786 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1154/2267] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:00:59 2022] DELETE /identity/v3/users/a66d0ec3a156469980551f9bc9034d4e => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:01:00.145691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91da966-fcfe-4f66-b685-1b9e478daa54 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:00.147780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a91da966-fcfe-4f66-b685-1b9e478daa54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:00.208791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91da966-fcfe-4f66-b685-1b9e478daa54 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:00.260510 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cdb121-d7c3-4138-bd66-8c1f5c072ffe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:00.261038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdb121-d7c3-4138-bd66-8c1f5c072ffe service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:00.261316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdb121-d7c3-4138-bd66-8c1f5c072ffe service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:00.261581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cdb121-d7c3-4138-bd66-8c1f5c072ffe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:00.263437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d8cdb121-d7c3-4138-bd66-8c1f5c072ffe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:00.274309 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:00.290522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2e944-4871-417e-8574-c49fa60b46b0 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:00.301209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-12f2e944-4871-417e-8574-c49fa60b46b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:00.333105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdb121-d7c3-4138-bd66-8c1f5c072ffe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:00.334565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cdb121-d7c3-4138-bd66-8c1f5c072ffe service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:00.346419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1114/2268] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:00.380705 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f2e944-4871-417e-8574-c49fa60b46b0 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:00.385997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20fc64-c0b0-41bf-b86b-06769f360334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:00.386639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20fc64-c0b0-41bf-b86b-06769f360334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:00.386911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20fc64-c0b0-41bf-b86b-06769f360334 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:00.387340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20fc64-c0b0-41bf-b86b-06769f360334 admin admin] PATH_INFO: `/v3/projects/bb2e9dd680194d6e9b861fed73188ba6` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:00.388632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20fc64-c0b0-41bf-b86b-06769f360334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb2e9dd680194d6e9b861fed73188ba6)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:00.390636 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20fc64-c0b0-41bf-b86b-06769f360334 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:00.410595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:00.537242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0c20fc64-c0b0-41bf-b86b-06769f360334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb2e9dd680194d6e9b861fed73188ba6', 'request_id': 'req-0c20fc64-c0b0-41bf-b86b-06769f360334'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:00.672087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0c20fc64-c0b0-41bf-b86b-06769f360334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb2e9dd680194d6e9b861fed73188ba6', 'request_id': 'req-0c20fc64-c0b0-41bf-b86b-06769f360334'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:00.680789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0c20fc64-c0b0-41bf-b86b-06769f360334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb2e9dd680194d6e9b861fed73188ba6', 'request_id': 'req-0c20fc64-c0b0-41bf-b86b-06769f360334'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:00.693139 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0c20fc64-c0b0-41bf-b86b-06769f360334 admin admin] The token cache is being invalidate because project bb2e9dd680194d6e9b861fed73188ba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:01:00.693495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0c20fc64-c0b0-41bf-b86b-06769f360334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:00.694136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0c20fc64-c0b0-41bf-b86b-06769f360334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:00.695695 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1155/2269] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:01:00 2022] DELETE /identity/v3/projects/bb2e9dd680194d6e9b861fed73188ba6 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:01:00.701319 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d43e5-6469-4ee5-be91-670ed561577e None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:00.701927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d43e5-6469-4ee5-be91-670ed561577e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:00.702203 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d43e5-6469-4ee5-be91-670ed561577e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:00.723591 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-989d43e5-6469-4ee5-be91-670ed561577e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:01:00.724817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-989d43e5-6469-4ee5-be91-670ed561577e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:00.789134 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1115/2270] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:01:00.800563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3b318-8f8c-4b60-8d67-c04dd9d73335 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:00.802233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3b318-8f8c-4b60-8d67-c04dd9d73335 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:00.803909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3b318-8f8c-4b60-8d67-c04dd9d73335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:00.827964 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-aeb3b318-8f8c-4b60-8d67-c04dd9d73335 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:01:00.829165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-aeb3b318-8f8c-4b60-8d67-c04dd9d73335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:00.838048 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1156/2271] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:01:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:01:00.842104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d43e5-6469-4ee5-be91-670ed561577e None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:00.852128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d43e5-6469-4ee5-be91-670ed561577e None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:00.857112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad03626-0feb-48cf-8c48-264825db2ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:00.857762 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad03626-0feb-48cf-8c48-264825db2ccb admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:00.858255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad03626-0feb-48cf-8c48-264825db2ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:00.858549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad03626-0feb-48cf-8c48-264825db2ccb admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:00.859178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad03626-0feb-48cf-8c48-264825db2ccb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:01:00.859490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad03626-0feb-48cf-8c48-264825db2ccb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:00.861150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad03626-0feb-48cf-8c48-264825db2ccb admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:00.876466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1116/2272] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:01:00.880854 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3b318-8f8c-4b60-8d67-c04dd9d73335 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:00.891843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3b318-8f8c-4b60-8d67-c04dd9d73335 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:00.896984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059aa5af-013e-4dbb-a7f3-f9875751f883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:00.897593 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059aa5af-013e-4dbb-a7f3-f9875751f883 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:00.897897 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059aa5af-013e-4dbb-a7f3-f9875751f883 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:00.898164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059aa5af-013e-4dbb-a7f3-f9875751f883 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:00.898823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059aa5af-013e-4dbb-a7f3-f9875751f883 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:00.900864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059aa5af-013e-4dbb-a7f3-f9875751f883 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:00.995114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1157/2273] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:01:00 2022] POST /identity/v3/projects => generated 391 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:01:00.999247 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad03626-0feb-48cf-8c48-264825db2ccb admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:01.010191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad03626-0feb-48cf-8c48-264825db2ccb admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:01.073742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1728f-70da-4f56-a7e4-3f4b98ebd247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:01.074375 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1728f-70da-4f56-a7e4-3f4b98ebd247 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:01.074660 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1728f-70da-4f56-a7e4-3f4b98ebd247 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:01.074930 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1728f-70da-4f56-a7e4-3f4b98ebd247 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:01.075571 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1728f-70da-4f56-a7e4-3f4b98ebd247 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:01.077527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1728f-70da-4f56-a7e4-3f4b98ebd247 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:01.258215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1117/2274] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:01:00 2022] POST /identity/v3/users => generated 296 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:01:01.259521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059aa5af-013e-4dbb-a7f3-f9875751f883 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:01.270528 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059aa5af-013e-4dbb-a7f3-f9875751f883 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:01.276044 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f792123-19d4-42ac-a3c7-79b22d799ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:01.276698 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f792123-19d4-42ac-a3c7-79b22d799ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:01.276978 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f792123-19d4-42ac-a3c7-79b22d799ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:01.277325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f792123-19d4-42ac-a3c7-79b22d799ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:01.277921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f792123-19d4-42ac-a3c7-79b22d799ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:01.279548 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f792123-19d4-42ac-a3c7-79b22d799ca8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:01.293547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1158/2275] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:01:01.296287 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1728f-70da-4f56-a7e4-3f4b98ebd247 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:01.307821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1728f-70da-4f56-a7e4-3f4b98ebd247 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:01.313185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd0d21-17f7-40a9-b793-3c284095ecea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:01.313846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd0d21-17f7-40a9-b793-3c284095ecea admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:01.314128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd0d21-17f7-40a9-b793-3c284095ecea admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:01.314400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd0d21-17f7-40a9-b793-3c284095ecea admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:01.315051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd0d21-17f7-40a9-b793-3c284095ecea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:01.316674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd0d21-17f7-40a9-b793-3c284095ecea admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:01.329621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1118/2276] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:01:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:01:01.333901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f792123-19d4-42ac-a3c7-79b22d799ca8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:01.343972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f792123-19d4-42ac-a3c7-79b22d799ca8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:01.348637 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1773ba1-f6ba-4c6e-8562-65a66c62836d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:01.349366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1773ba1-f6ba-4c6e-8562-65a66c62836d admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:01.349646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1773ba1-f6ba-4c6e-8562-65a66c62836d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:01.349956 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1773ba1-f6ba-4c6e-8562-65a66c62836d admin admin] PATH_INFO: `/v3/projects/e5e1144447ef4625bd0a4841d6595867/users/0587ce83d88c44769f3ee5c317c2c800/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:01.365530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1773ba1-f6ba-4c6e-8562-65a66c62836d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e1144447ef4625bd0a4841d6595867, user_id=0587ce83d88c44769f3ee5c317c2c800, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:01.368792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1773ba1-f6ba-4c6e-8562-65a66c62836d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:01.506485 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1159/2277] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:01:01 2022] PUT /identity/v3/projects/e5e1144447ef4625bd0a4841d6595867/users/0587ce83d88c44769f3ee5c317c2c800/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:01:01.517344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b9cb5-6d70-4ad7-9086-99c720e0b144 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:01.517628 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b9cb5-6d70-4ad7-9086-99c720e0b144 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:01.517932 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b9cb5-6d70-4ad7-9086-99c720e0b144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:01.548000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b86b9cb5-6d70-4ad7-9086-99c720e0b144 None None] MFA Rules not processed for user `0587ce83d88c44769f3ee5c317c2c800`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:01:01.549154 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b86b9cb5-6d70-4ad7-9086-99c720e0b144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:01.554478 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:01.615372 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1119/2278] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:01:01 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:01:01.626298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1773ba1-f6ba-4c6e-8562-65a66c62836d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:01.628041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a1773ba1-f6ba-4c6e-8562-65a66c62836d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:01.686662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1773ba1-f6ba-4c6e-8562-65a66c62836d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:01.690540 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e7828-f61e-4686-a461-ebdb84f11636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:01.691201 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e7828-f61e-4686-a461-ebdb84f11636 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:01.691481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e7828-f61e-4686-a461-ebdb84f11636 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:01.691753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6e7828-f61e-4686-a461-ebdb84f11636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:01.700642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e7828-f61e-4686-a461-ebdb84f11636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:01.702151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6e7828-f61e-4686-a461-ebdb84f11636 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:01.711244 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1160/2279] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:03.225786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b9cb5-6d70-4ad7-9086-99c720e0b144 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:03.225786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b86b9cb5-6d70-4ad7-9086-99c720e0b144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:03.305836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b9cb5-6d70-4ad7-9086-99c720e0b144 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:03.311213 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18bb8b-3250-4f87-873b-4c900cb0daec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:03.311847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18bb8b-3250-4f87-873b-4c900cb0daec service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:03.312159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18bb8b-3250-4f87-873b-4c900cb0daec service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:03.312460 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18bb8b-3250-4f87-873b-4c900cb0daec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:03.314250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-7e18bb8b-3250-4f87-873b-4c900cb0daec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:03.327615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18bb8b-3250-4f87-873b-4c900cb0daec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:03.327801 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18bb8b-3250-4f87-873b-4c900cb0daec service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:03.338245 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1120/2280] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:04.207469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e7828-f61e-4686-a461-ebdb84f11636 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:04.208919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3f6e7828-f61e-4686-a461-ebdb84f11636 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:04.259901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:04.291796 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6e7828-f61e-4686-a461-ebdb84f11636 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:04.303813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a6177-53d0-4a13-8b18-93f475aff8f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:04.304536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a6177-53d0-4a13-8b18-93f475aff8f2 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:04.304821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a6177-53d0-4a13-8b18-93f475aff8f2 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:04.305180 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a6177-53d0-4a13-8b18-93f475aff8f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:04.311992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2d0a6177-53d0-4a13-8b18-93f475aff8f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:04.334917 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a6177-53d0-4a13-8b18-93f475aff8f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:04.337297 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a6177-53d0-4a13-8b18-93f475aff8f2 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:04.356109 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1161/2281] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:07.969593 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18bb8b-3250-4f87-873b-4c900cb0daec service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:07.969593 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-7e18bb8b-3250-4f87-873b-4c900cb0daec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:07.983319 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18bb8b-3250-4f87-873b-4c900cb0daec service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:07.989574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3c0c41-17b4-4cbf-8333-ae1d2c6524b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:07.990347 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c0c41-17b4-4cbf-8333-ae1d2c6524b9 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:07.990661 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c0c41-17b4-4cbf-8333-ae1d2c6524b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:07.991049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3c0c41-17b4-4cbf-8333-ae1d2c6524b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:07.993176 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ae3c0c41-17b4-4cbf-8333-ae1d2c6524b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:08.068208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c0c41-17b4-4cbf-8333-ae1d2c6524b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:08.069691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3c0c41-17b4-4cbf-8333-ae1d2c6524b9 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:08.086939 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1121/2282] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:10.700331 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a6177-53d0-4a13-8b18-93f475aff8f2 service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:10.712320 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a6177-53d0-4a13-8b18-93f475aff8f2 service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:10.718523 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a801d2f-7496-4d40-8748-946a0e835476 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:10.719214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a801d2f-7496-4d40-8748-946a0e835476 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:10.719565 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a801d2f-7496-4d40-8748-946a0e835476 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:10.719843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a801d2f-7496-4d40-8748-946a0e835476 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:10.721560 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4a801d2f-7496-4d40-8748-946a0e835476 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:10.734499 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a801d2f-7496-4d40-8748-946a0e835476 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:10.736087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a801d2f-7496-4d40-8748-946a0e835476 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:10.746765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1162/2283] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:15.788776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55230ab2-2791-46a9-ab27-d6de37e67f85 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:15.789249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55230ab2-2791-46a9-ab27-d6de37e67f85 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:15.789361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55230ab2-2791-46a9-ab27-d6de37e67f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:15.808711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-55230ab2-2791-46a9-ab27-d6de37e67f85 None None] MFA Rules not processed for user `268dbf54f29c40d9a4daa772c44735d4`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:01:15.809934 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-55230ab2-2791-46a9-ab27-d6de37e67f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:15.852568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:15.864189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1122/2284] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:01:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:01:16.419256 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a801d2f-7496-4d40-8748-946a0e835476 service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:16.420722 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4a801d2f-7496-4d40-8748-946a0e835476 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:16.432380 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a801d2f-7496-4d40-8748-946a0e835476 service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:16.437311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762d2d8-f7a3-4274-ace9-a435057a3062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:16.437311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d2d8-f7a3-4274-ace9-a435057a3062 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:16.437311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d2d8-f7a3-4274-ace9-a435057a3062 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:16.437311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762d2d8-f7a3-4274-ace9-a435057a3062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:16.438702 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f762d2d8-f7a3-4274-ace9-a435057a3062 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:16.448424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762d2d8-f7a3-4274-ace9-a435057a3062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:16.449852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f762d2d8-f7a3-4274-ace9-a435057a3062 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:16.457883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1163/2285] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:22.975691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55230ab2-2791-46a9-ab27-d6de37e67f85 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:23.040459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55230ab2-2791-46a9-ab27-d6de37e67f85 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:23.045184 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ad3c-473d-4554-83b9-f4d4c50340b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:23.046345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ad3c-473d-4554-83b9-f4d4c50340b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:23.046345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ad3c-473d-4554-83b9-f4d4c50340b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:23.046345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ad3c-473d-4554-83b9-f4d4c50340b1 admin admin] PATH_INFO: `/v3/users/0587ce83d88c44769f3ee5c317c2c800` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:23.047469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900ad3c-473d-4554-83b9-f4d4c50340b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0587ce83d88c44769f3ee5c317c2c800)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:23.049400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900ad3c-473d-4554-83b9-f4d4c50340b1 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:23.251887 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e900ad3c-473d-4554-83b9-f4d4c50340b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0587ce83d88c44769f3ee5c317c2c800'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:23.264647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e900ad3c-473d-4554-83b9-f4d4c50340b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0587ce83d88c44769f3ee5c317c2c800'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:23.311794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e900ad3c-473d-4554-83b9-f4d4c50340b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0587ce83d88c44769f3ee5c317c2c800'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:23.312402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e900ad3c-473d-4554-83b9-f4d4c50340b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0587ce83d88c44769f3ee5c317c2c800'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:23.321393 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-e900ad3c-473d-4554-83b9-f4d4c50340b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0587ce83d88c44769f3ee5c317c2c800'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:23.327894 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1123/2286] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:01:22 2022] DELETE /identity/v3/users/0587ce83d88c44769f3ee5c317c2c800 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:01:23.332895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8700d-a266-40f3-9774-d0d42e31cf3d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:23.333373 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8700d-a266-40f3-9774-d0d42e31cf3d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:23.333710 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8700d-a266-40f3-9774-d0d42e31cf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:23.362960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0fe8700d-a266-40f3-9774-d0d42e31cf3d None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:01:23.364554 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0fe8700d-a266-40f3-9774-d0d42e31cf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:23.498846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1164/2287] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:01:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:01:23.502897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ad3c-473d-4554-83b9-f4d4c50340b1 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:23.504333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e900ad3c-473d-4554-83b9-f4d4c50340b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:23.587105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ad3c-473d-4554-83b9-f4d4c50340b1 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:23.595806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb073346-ccbf-4bfa-8a91-0f5115783e61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:23.595806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb073346-ccbf-4bfa-8a91-0f5115783e61 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:23.595806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb073346-ccbf-4bfa-8a91-0f5115783e61 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:23.595806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb073346-ccbf-4bfa-8a91-0f5115783e61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:23.605121 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb073346-ccbf-4bfa-8a91-0f5115783e61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:23.606275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb073346-ccbf-4bfa-8a91-0f5115783e61 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:23.616779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1124/2288] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:24.494368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8700d-a266-40f3-9774-d0d42e31cf3d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:24.496847 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0fe8700d-a266-40f3-9774-d0d42e31cf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:24.512004 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8700d-a266-40f3-9774-d0d42e31cf3d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:24.518123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:24.518285 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:24.518579 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:24.519003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] PATH_INFO: `/v3/projects/e5e1144447ef4625bd0a4841d6595867` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:24.520242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e1144447ef4625bd0a4841d6595867)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:24.522284 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:24.866782 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e1144447ef4625bd0a4841d6595867', 'request_id': 'req-7f6e10c2-801e-42f2-afd1-3975a8d249f8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:25.009424 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e1144447ef4625bd0a4841d6595867', 'request_id': 'req-7f6e10c2-801e-42f2-afd1-3975a8d249f8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:25.017258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e1144447ef4625bd0a4841d6595867', 'request_id': 'req-7f6e10c2-801e-42f2-afd1-3975a8d249f8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:25.029502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] The token cache is being invalidate because project e5e1144447ef4625bd0a4841d6595867 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:01:25.029788 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:25.030467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:25.031997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1165/2289] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:01:24 2022] DELETE /identity/v3/projects/e5e1144447ef4625bd0a4841d6595867 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:01:28.945947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb073346-ccbf-4bfa-8a91-0f5115783e61 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:28.947439 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-eb073346-ccbf-4bfa-8a91-0f5115783e61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:28.968254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:29.041292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb073346-ccbf-4bfa-8a91-0f5115783e61 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:29.108209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12833367-7570-400b-b296-88521b33bd2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:29.108816 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12833367-7570-400b-b296-88521b33bd2a service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:29.109090 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12833367-7570-400b-b296-88521b33bd2a service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:29.109358 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12833367-7570-400b-b296-88521b33bd2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:29.111751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-12833367-7570-400b-b296-88521b33bd2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:29.124041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12833367-7570-400b-b296-88521b33bd2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:29.125611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12833367-7570-400b-b296-88521b33bd2a service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:29.136042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1125/2290] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:35.025949 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:36.545157 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:36.546647 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:36.562480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6e10c2-801e-42f2-afd1-3975a8d249f8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:36.567973 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18334b-8bc7-4c74-a4cb-7629f419df51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:36.568570 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18334b-8bc7-4c74-a4cb-7629f419df51 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:36.568847 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18334b-8bc7-4c74-a4cb-7629f419df51 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:36.569119 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18334b-8bc7-4c74-a4cb-7629f419df51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:36.570812 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-de18334b-8bc7-4c74-a4cb-7629f419df51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:36.628749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18334b-8bc7-4c74-a4cb-7629f419df51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:36.630390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18334b-8bc7-4c74-a4cb-7629f419df51 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:36.639714 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1166/2291] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:39.131620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=55 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:40.017181 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12833367-7570-400b-b296-88521b33bd2a service nova] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:40.018600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-12833367-7570-400b-b296-88521b33bd2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:40.075665 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12833367-7570-400b-b296-88521b33bd2a service nova] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:40.079200 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9c852-310f-4352-aada-770c34ece55c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:40.079798 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9c852-310f-4352-aada-770c34ece55c service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:40.080074 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9c852-310f-4352-aada-770c34ece55c service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:40.080343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9c852-310f-4352-aada-770c34ece55c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:40.082069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d1a9c852-310f-4352-aada-770c34ece55c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:40.143971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9c852-310f-4352-aada-770c34ece55c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:40.145392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9c852-310f-4352-aada-770c34ece55c service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:40.155175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1126/2292] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:40.217739 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18334b-8bc7-4c74-a4cb-7629f419df51 service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:40.227100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18334b-8bc7-4c74-a4cb-7629f419df51 service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:40.231587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b093e-8dab-4f79-a3a2-85a5f7cad297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:40.232253 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b093e-8dab-4f79-a3a2-85a5f7cad297 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:40.232534 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b093e-8dab-4f79-a3a2-85a5f7cad297 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:40.232806 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b093e-8dab-4f79-a3a2-85a5f7cad297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:40.234635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fb1b093e-8dab-4f79-a3a2-85a5f7cad297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:40.298634 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b093e-8dab-4f79-a3a2-85a5f7cad297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:40.300155 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b093e-8dab-4f79-a3a2-85a5f7cad297 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:40.310284 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1167/2293] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:45.085481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9c852-310f-4352-aada-770c34ece55c service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:45.087049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d1a9c852-310f-4352-aada-770c34ece55c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:45.170184 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9c852-310f-4352-aada-770c34ece55c service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:45.175601 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77a667-c296-42ed-9146-5ba5e6bb9634 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03bc280d0395479981b7c7b08c96001b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:45.176200 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77a667-c296-42ed-9146-5ba5e6bb9634 service placement] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:45.176473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77a667-c296-42ed-9146-5ba5e6bb9634 service placement] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:45.176743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77a667-c296-42ed-9146-5ba5e6bb9634 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:45.178802 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-fe77a667-c296-42ed-9146-5ba5e6bb9634 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:45.190733 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77a667-c296-42ed-9146-5ba5e6bb9634 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:45.192467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77a667-c296-42ed-9146-5ba5e6bb9634 service placement] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:45.202995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1127/2294] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:45.245599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b093e-8dab-4f79-a3a2-85a5f7cad297 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:45.247236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fb1b093e-8dab-4f79-a3a2-85a5f7cad297 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:45.269579 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b093e-8dab-4f79-a3a2-85a5f7cad297 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:45.274936 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8411ac9-756f-429d-8719-a9da67db82a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:45.275467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8411ac9-756f-429d-8719-a9da67db82a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:45.275743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8411ac9-756f-429d-8719-a9da67db82a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:45.276107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8411ac9-756f-429d-8719-a9da67db82a0 admin admin] PATH_INFO: `/v3/users/766dd5a650fc4855980250969854f08d` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:45.277820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8411ac9-756f-429d-8719-a9da67db82a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=766dd5a650fc4855980250969854f08d)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:45.279800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8411ac9-756f-429d-8719-a9da67db82a0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:45.444782 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8411ac9-756f-429d-8719-a9da67db82a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '766dd5a650fc4855980250969854f08d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:45.551618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8411ac9-756f-429d-8719-a9da67db82a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '766dd5a650fc4855980250969854f08d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:45.564491 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8411ac9-756f-429d-8719-a9da67db82a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '766dd5a650fc4855980250969854f08d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:45.565160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8411ac9-756f-429d-8719-a9da67db82a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '766dd5a650fc4855980250969854f08d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:45.565765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d8411ac9-756f-429d-8719-a9da67db82a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '766dd5a650fc4855980250969854f08d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:45.578697 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1168/2295] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:01:45 2022] DELETE /identity/v3/users/766dd5a650fc4855980250969854f08d => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:01:45.845524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77a667-c296-42ed-9146-5ba5e6bb9634 service placement] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:45.857212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77a667-c296-42ed-9146-5ba5e6bb9634 service placement] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:45.908506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:45.926580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:45.927343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:45.927627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:45.927897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e admin admin] PATH_INFO: `/v3/projects/e1dc64fa9e8e422fab6b3a66a29278ff` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:45.929006 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1dc64fa9e8e422fab6b3a66a29278ff)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:45.930941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:46.105897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1dc64fa9e8e422fab6b3a66a29278ff', 'request_id': 'req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:46.111351 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1dc64fa9e8e422fab6b3a66a29278ff', 'request_id': 'req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:46.117817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1dc64fa9e8e422fab6b3a66a29278ff', 'request_id': 'req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:46.225327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e admin admin] The token cache is being invalidate because project e1dc64fa9e8e422fab6b3a66a29278ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:01:46.225613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:46.226250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d85d10a0-5d2b-4a23-8dd5-61af7d2a8e2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:46.227717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1128/2296] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:01:45 2022] DELETE /identity/v3/projects/e1dc64fa9e8e422fab6b3a66a29278ff => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:01:46.232855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a96d18-3029-4636-9d1e-5e78b9aa6eab None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:46.237049 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a96d18-3029-4636-9d1e-5e78b9aa6eab None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:46.237049 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a96d18-3029-4636-9d1e-5e78b9aa6eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:46.241907 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:46.253140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c6a96d18-3029-4636-9d1e-5e78b9aa6eab None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:01:46.254410 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c6a96d18-3029-4636-9d1e-5e78b9aa6eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:46.315975 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1169/2297] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:01:46.324660 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c897ac-32b4-49c3-b12a-f77f4d5b47d6 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:46.324919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c897ac-32b4-49c3-b12a-f77f4d5b47d6 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:46.333106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c897ac-32b4-49c3-b12a-f77f4d5b47d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:46.346638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-d4c897ac-32b4-49c3-b12a-f77f4d5b47d6 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:01:46.348349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d4c897ac-32b4-49c3-b12a-f77f4d5b47d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:46.356951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1129/2298] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:01:46.361166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a96d18-3029-4636-9d1e-5e78b9aa6eab None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:46.372222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a96d18-3029-4636-9d1e-5e78b9aa6eab None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:46.377023 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0eb018-c0de-4470-a79b-302692eb0f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:46.377660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0eb018-c0de-4470-a79b-302692eb0f07 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:46.377982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0eb018-c0de-4470-a79b-302692eb0f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:46.378254 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0eb018-c0de-4470-a79b-302692eb0f07 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:46.379087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0eb018-c0de-4470-a79b-302692eb0f07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:01:46.379384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0eb018-c0de-4470-a79b-302692eb0f07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:46.382087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0eb018-c0de-4470-a79b-302692eb0f07 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:46.394816 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1170/2299] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:01:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:01:46.399339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c897ac-32b4-49c3-b12a-f77f4d5b47d6 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:46.409172 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c897ac-32b4-49c3-b12a-f77f4d5b47d6 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:46.414335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d236e-6b5c-4418-8986-b71d689a7902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:46.414973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d236e-6b5c-4418-8986-b71d689a7902 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:46.415198 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d236e-6b5c-4418-8986-b71d689a7902 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:46.415496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711d236e-6b5c-4418-8986-b71d689a7902 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:46.416311 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d236e-6b5c-4418-8986-b71d689a7902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:46.418344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711d236e-6b5c-4418-8986-b71d689a7902 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:46.524492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1130/2300] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:01:46 2022] POST /identity/v3/projects => generated 363 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:01:46.529305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0eb018-c0de-4470-a79b-302692eb0f07 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:46.539194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0eb018-c0de-4470-a79b-302692eb0f07 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:46.594145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca09ba-2ec2-46b8-bf55-e9df8b6ab9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:46.594743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca09ba-2ec2-46b8-bf55-e9df8b6ab9e0 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:46.595019 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca09ba-2ec2-46b8-bf55-e9df8b6ab9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:46.595288 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ca09ba-2ec2-46b8-bf55-e9df8b6ab9e0 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:46.595903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca09ba-2ec2-46b8-bf55-e9df8b6ab9e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:46.598375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ca09ba-2ec2-46b8-bf55-e9df8b6ab9e0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:46.751207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1171/2301] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:01:46 2022] POST /identity/v3/users => generated 282 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:01:46.755243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d236e-6b5c-4418-8986-b71d689a7902 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:46.765967 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711d236e-6b5c-4418-8986-b71d689a7902 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:46.770512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44925adb-a91c-4add-b93a-09a08557b2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:46.771098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44925adb-a91c-4add-b93a-09a08557b2da admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:46.771986 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44925adb-a91c-4add-b93a-09a08557b2da admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:46.772269 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44925adb-a91c-4add-b93a-09a08557b2da admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:46.772901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44925adb-a91c-4add-b93a-09a08557b2da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:46.774525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44925adb-a91c-4add-b93a-09a08557b2da admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:46.784716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1131/2302] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:01:46.792054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca09ba-2ec2-46b8-bf55-e9df8b6ab9e0 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:46.801791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ca09ba-2ec2-46b8-bf55-e9df8b6ab9e0 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:46.806571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efac398-b3d4-4c0a-9822-5680a1569b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:46.807195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efac398-b3d4-4c0a-9822-5680a1569b0d admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:46.807470 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efac398-b3d4-4c0a-9822-5680a1569b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:46.807740 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efac398-b3d4-4c0a-9822-5680a1569b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:46.808456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efac398-b3d4-4c0a-9822-5680a1569b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:46.810058 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efac398-b3d4-4c0a-9822-5680a1569b0d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:46.820571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1172/2303] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:01:46.824981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44925adb-a91c-4add-b93a-09a08557b2da admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:46.835326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44925adb-a91c-4add-b93a-09a08557b2da admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:46.840701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17839c4d-7c6f-4578-aad6-5b13fefe1dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:46.841335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17839c4d-7c6f-4578-aad6-5b13fefe1dbc admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:46.841608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17839c4d-7c6f-4578-aad6-5b13fefe1dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:46.841908 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17839c4d-7c6f-4578-aad6-5b13fefe1dbc admin admin] PATH_INFO: `/v3/projects/9dc51956a8b4497a9e4d811f2793f776/users/84cb4f2436214d0a8392ae4274eb78bd/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:46.856439 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17839c4d-7c6f-4578-aad6-5b13fefe1dbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dc51956a8b4497a9e4d811f2793f776, user_id=84cb4f2436214d0a8392ae4274eb78bd, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:46.859337 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17839c4d-7c6f-4578-aad6-5b13fefe1dbc admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:46.997930 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1132/2304] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:01:46 2022] PUT /identity/v3/projects/9dc51956a8b4497a9e4d811f2793f776/users/84cb4f2436214d0a8392ae4274eb78bd/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:01:47.008179 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088176-3377-4cb0-9923-3cdb2ca8f13a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:47.008396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088176-3377-4cb0-9923-3cdb2ca8f13a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:47.008662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed088176-3377-4cb0-9923-3cdb2ca8f13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:47.036915 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-ed088176-3377-4cb0-9923-3cdb2ca8f13a None None] MFA Rules not processed for user `84cb4f2436214d0a8392ae4274eb78bd`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:01:47.038133 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ed088176-3377-4cb0-9923-3cdb2ca8f13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:47.098489 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:47.101998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1173/2305] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:01:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:01:47.106568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17839c4d-7c6f-4578-aad6-5b13fefe1dbc admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:47.168696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17839c4d-7c6f-4578-aad6-5b13fefe1dbc admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:47.173279 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fb545-d5c5-4a91-9331-e4644f684db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:47.173893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb545-d5c5-4a91-9331-e4644f684db0 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:47.174171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb545-d5c5-4a91-9331-e4644f684db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:47.174442 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fb545-d5c5-4a91-9331-e4644f684db0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:47.175084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fb545-d5c5-4a91-9331-e4644f684db0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:47.176959 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fb545-d5c5-4a91-9331-e4644f684db0 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:47.188657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:47.313535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1133/2306] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:01:47 2022] POST /identity/v3/projects => generated 363 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:01:47.317681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed088176-3377-4cb0-9923-3cdb2ca8f13a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:47.328765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed088176-3377-4cb0-9923-3cdb2ca8f13a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:47.384957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7da7f-7892-4e93-865f-2e997763e03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:47.386237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7da7f-7892-4e93-865f-2e997763e03f admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:47.386569 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7da7f-7892-4e93-865f-2e997763e03f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:47.386840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7da7f-7892-4e93-865f-2e997763e03f admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:47.387497 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7da7f-7892-4e93-865f-2e997763e03f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:47.389418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7da7f-7892-4e93-865f-2e997763e03f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:47.513000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1174/2307] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:01:47 2022] POST /identity/v3/users => generated 288 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:01:47.517227 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fb545-d5c5-4a91-9331-e4644f684db0 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:47.527779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fb545-d5c5-4a91-9331-e4644f684db0 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:47.532499 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab12ef-2775-4974-937d-940f8b80951f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:47.533206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab12ef-2775-4974-937d-940f8b80951f admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:47.533458 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab12ef-2775-4974-937d-940f8b80951f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:47.533730 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbab12ef-2775-4974-937d-940f8b80951f admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:47.534498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab12ef-2775-4974-937d-940f8b80951f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:47.536092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbab12ef-2775-4974-937d-940f8b80951f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:47.549454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1134/2308] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:01:47.552204 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7da7f-7892-4e93-865f-2e997763e03f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:47.563092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7da7f-7892-4e93-865f-2e997763e03f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:47.568264 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76befa87-ec73-497f-bca6-2c198c85f553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:47.568853 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76befa87-ec73-497f-bca6-2c198c85f553 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:47.569158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76befa87-ec73-497f-bca6-2c198c85f553 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:47.569430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76befa87-ec73-497f-bca6-2c198c85f553 admin admin] PATH_INFO: `/v3/projects/ae3079c9bc2e4efd9ec41965017522a7/users/d61ef265b28e43a4b87ef0c447d35d78/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:47.584785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76befa87-ec73-497f-bca6-2c198c85f553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3079c9bc2e4efd9ec41965017522a7, user_id=d61ef265b28e43a4b87ef0c447d35d78, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:47.587579 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76befa87-ec73-497f-bca6-2c198c85f553 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:47.694970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1175/2309] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:01:47 2022] PUT /identity/v3/projects/ae3079c9bc2e4efd9ec41965017522a7/users/d61ef265b28e43a4b87ef0c447d35d78/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:01:47.698981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab12ef-2775-4974-937d-940f8b80951f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:47.709213 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbab12ef-2775-4974-937d-940f8b80951f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:47.763712 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9f34f8-a73a-45af-9323-ee8baf1dc217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:47.764302 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f34f8-a73a-45af-9323-ee8baf1dc217 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:47.764579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f34f8-a73a-45af-9323-ee8baf1dc217 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:47.764849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9f34f8-a73a-45af-9323-ee8baf1dc217 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:47.765474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f34f8-a73a-45af-9323-ee8baf1dc217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:47.767092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9f34f8-a73a-45af-9323-ee8baf1dc217 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:47.776433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1135/2310] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:01:47.780671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76befa87-ec73-497f-bca6-2c198c85f553 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:47.790836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76befa87-ec73-497f-bca6-2c198c85f553 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:47.796004 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc557c-159f-4c4a-a81e-8e11d934795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:47.796649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc557c-159f-4c4a-a81e-8e11d934795e admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:47.796928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc557c-159f-4c4a-a81e-8e11d934795e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:47.797200 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc557c-159f-4c4a-a81e-8e11d934795e admin admin] PATH_INFO: `/v3/projects/ae3079c9bc2e4efd9ec41965017522a7/users/d61ef265b28e43a4b87ef0c447d35d78/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:47.799236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc557c-159f-4c4a-a81e-8e11d934795e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3079c9bc2e4efd9ec41965017522a7, user_id=d61ef265b28e43a4b87ef0c447d35d78, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:47.802052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc557c-159f-4c4a-a81e-8e11d934795e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:47.813349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1176/2311] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:01:47 2022] PUT /identity/v3/projects/ae3079c9bc2e4efd9ec41965017522a7/users/d61ef265b28e43a4b87ef0c447d35d78/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:01:47.829612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bc280-d1c2-412d-ac9e-fb9676ee4db4 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:47.829612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bc280-d1c2-412d-ac9e-fb9676ee4db4 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:47.829612 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bc280-d1c2-412d-ac9e-fb9676ee4db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:47.851329 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-aa3bc280-d1c2-412d-ac9e-fb9676ee4db4 None None] MFA Rules not processed for user `d61ef265b28e43a4b87ef0c447d35d78`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:01:47.852519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-aa3bc280-d1c2-412d-ac9e-fb9676ee4db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:47.923488 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1136/2312] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:01:47 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:01:47.934772 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc557c-159f-4c4a-a81e-8e11d934795e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:47.936443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ccfc557c-159f-4c4a-a81e-8e11d934795e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:47.992402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc557c-159f-4c4a-a81e-8e11d934795e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:47.996823 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533bd60-c40a-4fae-99e4-76c1260f0c7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:47.997422 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533bd60-c40a-4fae-99e4-76c1260f0c7b service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:47.997696 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533bd60-c40a-4fae-99e4-76c1260f0c7b service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:47.998017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533bd60-c40a-4fae-99e4-76c1260f0c7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:48.008163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533bd60-c40a-4fae-99e4-76c1260f0c7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:48.009576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533bd60-c40a-4fae-99e4-76c1260f0c7b service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:48.020390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1177/2313] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:47 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:49.099737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bc280-d1c2-412d-ac9e-fb9676ee4db4 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:49.159579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3bc280-d1c2-412d-ac9e-fb9676ee4db4 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:49.164122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6e916-7ddd-4cfc-ae45-d2356bd6e355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:49.164730 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6e916-7ddd-4cfc-ae45-d2356bd6e355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:49.165005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6e916-7ddd-4cfc-ae45-d2356bd6e355 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:49.165272 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c6e916-7ddd-4cfc-ae45-d2356bd6e355 admin admin] PATH_INFO: `/v3/users/84cb4f2436214d0a8392ae4274eb78bd` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:49.166444 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c6e916-7ddd-4cfc-ae45-d2356bd6e355 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84cb4f2436214d0a8392ae4274eb78bd)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:49.168832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c6e916-7ddd-4cfc-ae45-d2356bd6e355 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:49.171097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:49.348845 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-04c6e916-7ddd-4cfc-ae45-d2356bd6e355 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84cb4f2436214d0a8392ae4274eb78bd'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:49.361021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-04c6e916-7ddd-4cfc-ae45-d2356bd6e355 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84cb4f2436214d0a8392ae4274eb78bd'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:49.437745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-04c6e916-7ddd-4cfc-ae45-d2356bd6e355 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84cb4f2436214d0a8392ae4274eb78bd'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:49.438355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-04c6e916-7ddd-4cfc-ae45-d2356bd6e355 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84cb4f2436214d0a8392ae4274eb78bd'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:49.445839 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-04c6e916-7ddd-4cfc-ae45-d2356bd6e355 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84cb4f2436214d0a8392ae4274eb78bd'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:49.451750 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1137/2314] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:01:49 2022] DELETE /identity/v3/users/84cb4f2436214d0a8392ae4274eb78bd => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:01:49.456273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476e072-169d-4265-9383-3a92967f1010 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:49.456825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476e072-169d-4265-9383-3a92967f1010 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:49.456825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8476e072-169d-4265-9383-3a92967f1010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:49.476810 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8476e072-169d-4265-9383-3a92967f1010 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:01:49.478085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8476e072-169d-4265-9383-3a92967f1010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:49.524885 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:49.536439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1178/2315] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:01:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:01:49.546806 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6e916-7ddd-4cfc-ae45-d2356bd6e355 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:49.548229 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-04c6e916-7ddd-4cfc-ae45-d2356bd6e355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:49.604105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c6e916-7ddd-4cfc-ae45-d2356bd6e355 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:49.607832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9a81d-f603-4d2e-afea-b759dd03a350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:49.608422 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9a81d-f603-4d2e-afea-b759dd03a350 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:49.608699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9a81d-f603-4d2e-afea-b759dd03a350 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:49.608972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb9a81d-f603-4d2e-afea-b759dd03a350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:49.617752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9a81d-f603-4d2e-afea-b759dd03a350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:49.619189 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb9a81d-f603-4d2e-afea-b759dd03a350 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:49.628918 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1138/2316] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:01:50.107484 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476e072-169d-4265-9383-3a92967f1010 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:50.108955 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8476e072-169d-4265-9383-3a92967f1010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:50.124003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8476e072-169d-4265-9383-3a92967f1010 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:50.128932 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:50.129680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:50.130057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:50.130399 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] PATH_INFO: `/v3/projects/9dc51956a8b4497a9e4d811f2793f776` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:50.131626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dc51956a8b4497a9e4d811f2793f776)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:50.133429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:50.225442 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dc51956a8b4497a9e4d811f2793f776', 'request_id': 'req-168eff12-07c1-4cd6-a225-dd4ead018e6c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:50.305879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dc51956a8b4497a9e4d811f2793f776', 'request_id': 'req-168eff12-07c1-4cd6-a225-dd4ead018e6c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:50.314217 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dc51956a8b4497a9e4d811f2793f776', 'request_id': 'req-168eff12-07c1-4cd6-a225-dd4ead018e6c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:50.330072 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] The token cache is being invalidate because project 9dc51956a8b4497a9e4d811f2793f776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:01:50.330368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:50.331017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:50.332519 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1179/2317] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:01:50 2022] DELETE /identity/v3/projects/9dc51956a8b4497a9e4d811f2793f776 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:01:50.336953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9a81d-f603-4d2e-afea-b759dd03a350 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:50.347022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb9a81d-f603-4d2e-afea-b759dd03a350 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:50.405734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6ec67-e7e7-471d-9068-93e5c18481ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:50.406401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6ec67-e7e7-471d-9068-93e5c18481ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:50.406817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6ec67-e7e7-471d-9068-93e5c18481ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:50.407096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6ec67-e7e7-471d-9068-93e5c18481ff admin admin] PATH_INFO: `/v3/users/d61ef265b28e43a4b87ef0c447d35d78` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:50.408208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6ec67-e7e7-471d-9068-93e5c18481ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d61ef265b28e43a4b87ef0c447d35d78)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:50.410150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6ec67-e7e7-471d-9068-93e5c18481ff admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:50.518143 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:50.650862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d9d6ec67-e7e7-471d-9068-93e5c18481ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd61ef265b28e43a4b87ef0c447d35d78'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:50.662712 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d9d6ec67-e7e7-471d-9068-93e5c18481ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd61ef265b28e43a4b87ef0c447d35d78'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:50.734715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d9d6ec67-e7e7-471d-9068-93e5c18481ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd61ef265b28e43a4b87ef0c447d35d78'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:50.735337 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d9d6ec67-e7e7-471d-9068-93e5c18481ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd61ef265b28e43a4b87ef0c447d35d78'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:50.742772 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d9d6ec67-e7e7-471d-9068-93e5c18481ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd61ef265b28e43a4b87ef0c447d35d78'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:50.749321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1139/2318] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:01:50 2022] DELETE /identity/v3/users/d61ef265b28e43a4b87ef0c447d35d78 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:01:51.175353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:51.177185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:51.212706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168eff12-07c1-4cd6-a225-dd4ead018e6c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:51.275132 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84081f7c-9d31-4101-b112-33f9fb90ba71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:51.275911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84081f7c-9d31-4101-b112-33f9fb90ba71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:51.276912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84081f7c-9d31-4101-b112-33f9fb90ba71 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:51.277193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84081f7c-9d31-4101-b112-33f9fb90ba71 admin admin] PATH_INFO: `/v3/projects/ae3079c9bc2e4efd9ec41965017522a7` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:51.278373 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84081f7c-9d31-4101-b112-33f9fb90ba71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae3079c9bc2e4efd9ec41965017522a7)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:51.280282 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84081f7c-9d31-4101-b112-33f9fb90ba71 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:51.426607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84081f7c-9d31-4101-b112-33f9fb90ba71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae3079c9bc2e4efd9ec41965017522a7', 'request_id': 'req-84081f7c-9d31-4101-b112-33f9fb90ba71'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:51.500500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84081f7c-9d31-4101-b112-33f9fb90ba71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae3079c9bc2e4efd9ec41965017522a7', 'request_id': 'req-84081f7c-9d31-4101-b112-33f9fb90ba71'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:51.507738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84081f7c-9d31-4101-b112-33f9fb90ba71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae3079c9bc2e4efd9ec41965017522a7', 'request_id': 'req-84081f7c-9d31-4101-b112-33f9fb90ba71'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:51.518743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84081f7c-9d31-4101-b112-33f9fb90ba71 admin admin] The token cache is being invalidate because project ae3079c9bc2e4efd9ec41965017522a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:01:51.519029 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84081f7c-9d31-4101-b112-33f9fb90ba71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:51.519648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84081f7c-9d31-4101-b112-33f9fb90ba71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:01:51.521111 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1180/2319] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:01:51 2022] DELETE /identity/v3/projects/ae3079c9bc2e4efd9ec41965017522a7 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:01:51.526198 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cef65-f23a-408b-b9eb-9c86065292c2 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:51.527242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cef65-f23a-408b-b9eb-9c86065292c2 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:51.527242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cef65-f23a-408b-b9eb-9c86065292c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:51.546887 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-6a9cef65-f23a-408b-b9eb-9c86065292c2 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:01:51.548143 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6a9cef65-f23a-408b-b9eb-9c86065292c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:51.607327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1140/2320] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:01:51.616295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c22b9-f430-4524-b5f8-5de3df8c1cb2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:51.616578 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c22b9-f430-4524-b5f8-5de3df8c1cb2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:51.616851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c22b9-f430-4524-b5f8-5de3df8c1cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:51.637531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-bb0c22b9-f430-4524-b5f8-5de3df8c1cb2 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:01:51.638731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bb0c22b9-f430-4524-b5f8-5de3df8c1cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:51.646553 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1181/2321] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:01:51.658032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cef65-f23a-408b-b9eb-9c86065292c2 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:51.667588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cef65-f23a-408b-b9eb-9c86065292c2 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:51.672168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904ff0d-2db9-4dac-96f7-1f604ca1f9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:51.672754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904ff0d-2db9-4dac-96f7-1f604ca1f9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:51.673033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904ff0d-2db9-4dac-96f7-1f604ca1f9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:51.673304 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904ff0d-2db9-4dac-96f7-1f604ca1f9b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:51.673972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904ff0d-2db9-4dac-96f7-1f604ca1f9b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:01:51.674275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904ff0d-2db9-4dac-96f7-1f604ca1f9b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:51.675856 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904ff0d-2db9-4dac-96f7-1f604ca1f9b9 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:51.687649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1141/2322] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:01:51.691859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c22b9-f430-4524-b5f8-5de3df8c1cb2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:51.696317 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:51.702107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c22b9-f430-4524-b5f8-5de3df8c1cb2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:51.706742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10d5ff-8b2b-4413-b87e-b74450848146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:51.707328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10d5ff-8b2b-4413-b87e-b74450848146 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:51.707602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10d5ff-8b2b-4413-b87e-b74450848146 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:51.708136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac10d5ff-8b2b-4413-b87e-b74450848146 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:51.708745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10d5ff-8b2b-4413-b87e-b74450848146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:51.710700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac10d5ff-8b2b-4413-b87e-b74450848146 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:51.849759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1182/2323] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:01:51 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:01:51.854180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904ff0d-2db9-4dac-96f7-1f604ca1f9b9 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:51.864198 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904ff0d-2db9-4dac-96f7-1f604ca1f9b9 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:51.924864 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542caa2-ca87-41bf-bbfe-05920e590ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:51.925403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542caa2-ca87-41bf-bbfe-05920e590ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:51.925664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542caa2-ca87-41bf-bbfe-05920e590ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:51.925975 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a542caa2-ca87-41bf-bbfe-05920e590ea1 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:51.926827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542caa2-ca87-41bf-bbfe-05920e590ea1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:51.929056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a542caa2-ca87-41bf-bbfe-05920e590ea1 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:51.991456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1142/2324] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:01:51 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:01:51.996357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10d5ff-8b2b-4413-b87e-b74450848146 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:52.006927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac10d5ff-8b2b-4413-b87e-b74450848146 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:52.012204 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20087293-cd37-4aff-bdaa-65522e3b198a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:52.012811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087293-cd37-4aff-bdaa-65522e3b198a admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:52.013104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087293-cd37-4aff-bdaa-65522e3b198a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:52.013375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20087293-cd37-4aff-bdaa-65522e3b198a admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:52.014040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20087293-cd37-4aff-bdaa-65522e3b198a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:52.015639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20087293-cd37-4aff-bdaa-65522e3b198a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:52.025390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1183/2325] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:01:52.029360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542caa2-ca87-41bf-bbfe-05920e590ea1 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:52.039124 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a542caa2-ca87-41bf-bbfe-05920e590ea1 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:52.044279 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fb1c5-36bf-4af3-b039-49b8e6115d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:52.044862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fb1c5-36bf-4af3-b039-49b8e6115d7f admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:52.045140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fb1c5-36bf-4af3-b039-49b8e6115d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:52.045411 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fb1c5-36bf-4af3-b039-49b8e6115d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:52.046095 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fb1c5-36bf-4af3-b039-49b8e6115d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:52.047662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fb1c5-36bf-4af3-b039-49b8e6115d7f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:52.058671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1143/2326] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:01:52.062531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20087293-cd37-4aff-bdaa-65522e3b198a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:52.072175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20087293-cd37-4aff-bdaa-65522e3b198a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:52.076761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec4984-4221-4d74-9938-ee4e606804a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:52.077398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec4984-4221-4d74-9938-ee4e606804a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:52.077671 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec4984-4221-4d74-9938-ee4e606804a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:52.077976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec4984-4221-4d74-9938-ee4e606804a9 admin admin] PATH_INFO: `/v3/projects/9d68672014624b05a7115633aacea896/users/e7ef98a67a5349ecb7294857daaf203a/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:52.092286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec4984-4221-4d74-9938-ee4e606804a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d68672014624b05a7115633aacea896, user_id=e7ef98a67a5349ecb7294857daaf203a, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:52.094940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec4984-4221-4d74-9938-ee4e606804a9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:52.209230 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1184/2327] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:01:52 2022] PUT /identity/v3/projects/9d68672014624b05a7115633aacea896/users/e7ef98a67a5349ecb7294857daaf203a/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:01:52.219524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf51a8-837f-4256-ae32-8601240ac3b1 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:52.220013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf51a8-837f-4256-ae32-8601240ac3b1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:52.220364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf51a8-837f-4256-ae32-8601240ac3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:52.240632 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:52.249875 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-93cf51a8-837f-4256-ae32-8601240ac3b1 None None] MFA Rules not processed for user `e7ef98a67a5349ecb7294857daaf203a`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:01:52.251207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-93cf51a8-837f-4256-ae32-8601240ac3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:52.316349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1144/2328] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:01:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:01:52.321116 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec4984-4221-4d74-9938-ee4e606804a9 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:52.388888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec4984-4221-4d74-9938-ee4e606804a9 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:52.393599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3320f-bcbc-42c5-91c2-9a7bb9d580b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:52.394237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3320f-bcbc-42c5-91c2-9a7bb9d580b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:52.394514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3320f-bcbc-42c5-91c2-9a7bb9d580b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:52.394790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d3320f-bcbc-42c5-91c2-9a7bb9d580b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:52.398506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3320f-bcbc-42c5-91c2-9a7bb9d580b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:52.400421 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d3320f-bcbc-42c5-91c2-9a7bb9d580b6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:52.525617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1185/2329] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:01:52 2022] POST /identity/v3/projects => generated 377 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:01:52.530437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf51a8-837f-4256-ae32-8601240ac3b1 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:52.541123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf51a8-837f-4256-ae32-8601240ac3b1 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:52.598121 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d4944-8ef4-4b4f-a5fc-1bcd4a014d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:52.598716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d4944-8ef4-4b4f-a5fc-1bcd4a014d8f admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:52.598988 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d4944-8ef4-4b4f-a5fc-1bcd4a014d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:52.599252 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d4944-8ef4-4b4f-a5fc-1bcd4a014d8f admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:52.605281 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d4944-8ef4-4b4f-a5fc-1bcd4a014d8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:52.605281 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d4944-8ef4-4b4f-a5fc-1bcd4a014d8f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:53.020634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1145/2330] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:01:52 2022] POST /identity/v3/users => generated 295 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:01:53.024778 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3320f-bcbc-42c5-91c2-9a7bb9d580b6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:53.035443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d3320f-bcbc-42c5-91c2-9a7bb9d580b6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:53.039872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624df8b1-9c91-4411-aef3-b00cf3c387eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:53.040513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624df8b1-9c91-4411-aef3-b00cf3c387eb admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:53.040736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624df8b1-9c91-4411-aef3-b00cf3c387eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:53.041003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624df8b1-9c91-4411-aef3-b00cf3c387eb admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:53.041625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624df8b1-9c91-4411-aef3-b00cf3c387eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:53.043230 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624df8b1-9c91-4411-aef3-b00cf3c387eb admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:53.053481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1186/2331] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:01:53.057438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d4944-8ef4-4b4f-a5fc-1bcd4a014d8f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:53.066992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d4944-8ef4-4b4f-a5fc-1bcd4a014d8f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:53.071495 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b643e-84f9-4294-8a13-9ec52ccb1d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:53.072123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b643e-84f9-4294-8a13-9ec52ccb1d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:53.072396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b643e-84f9-4294-8a13-9ec52ccb1d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:53.072662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b643e-84f9-4294-8a13-9ec52ccb1d03 admin admin] PATH_INFO: `/v3/projects/d8a63a6079774d9eac59bb74ad415a12/users/21a11bc000dc4bd4b28316421e5f3cb8/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:53.089584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b643e-84f9-4294-8a13-9ec52ccb1d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a63a6079774d9eac59bb74ad415a12, user_id=21a11bc000dc4bd4b28316421e5f3cb8, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:53.092421 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b643e-84f9-4294-8a13-9ec52ccb1d03 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:53.197402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1146/2332] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:01:53 2022] PUT /identity/v3/projects/d8a63a6079774d9eac59bb74ad415a12/users/21a11bc000dc4bd4b28316421e5f3cb8/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:01:53.201875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624df8b1-9c91-4411-aef3-b00cf3c387eb admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:53.212226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624df8b1-9c91-4411-aef3-b00cf3c387eb admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:53.271876 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8de76-f297-4c5e-b479-ec4ab20d0582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:53.272488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8de76-f297-4c5e-b479-ec4ab20d0582 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:53.272769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8de76-f297-4c5e-b479-ec4ab20d0582 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:53.273041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8de76-f297-4c5e-b479-ec4ab20d0582 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:53.273668 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8de76-f297-4c5e-b479-ec4ab20d0582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:53.275361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a8de76-f297-4c5e-b479-ec4ab20d0582 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:53.284957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1187/2333] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:01:53.289120 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b643e-84f9-4294-8a13-9ec52ccb1d03 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:53.299741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b643e-84f9-4294-8a13-9ec52ccb1d03 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:53.304436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84a9c8-f69f-49f1-a198-4290c574fc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:53.305071 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84a9c8-f69f-49f1-a198-4290c574fc4b admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:53.305342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84a9c8-f69f-49f1-a198-4290c574fc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:53.305609 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c84a9c8-f69f-49f1-a198-4290c574fc4b admin admin] PATH_INFO: `/v3/projects/d8a63a6079774d9eac59bb74ad415a12/users/21a11bc000dc4bd4b28316421e5f3cb8/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:53.308025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84a9c8-f69f-49f1-a198-4290c574fc4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a63a6079774d9eac59bb74ad415a12, user_id=21a11bc000dc4bd4b28316421e5f3cb8, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:53.310676 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c84a9c8-f69f-49f1-a198-4290c574fc4b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:53.320566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1147/2334] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:01:53 2022] PUT /identity/v3/projects/d8a63a6079774d9eac59bb74ad415a12/users/21a11bc000dc4bd4b28316421e5f3cb8/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:01:53.331290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01243e-009c-46d4-a752-451e843391c2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:53.331573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01243e-009c-46d4-a752-451e843391c2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:53.331849 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01243e-009c-46d4-a752-451e843391c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:53.359069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-db01243e-009c-46d4-a752-451e843391c2 None None] MFA Rules not processed for user `21a11bc000dc4bd4b28316421e5f3cb8`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:01:53.360274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-db01243e-009c-46d4-a752-451e843391c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:53.364132 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:53.429731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1188/2335] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:01:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:01:53.441369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84a9c8-f69f-49f1-a198-4290c574fc4b admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:01:53.443239 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2c84a9c8-f69f-49f1-a198-4290c574fc4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:01:53.504184 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c84a9c8-f69f-49f1-a198-4290c574fc4b admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:01:53.507952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c3ec15-ee33-4566-b3bd-8aab3f91d29c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:01:53.508637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3ec15-ee33-4566-b3bd-8aab3f91d29c service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:01:53.508979 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3ec15-ee33-4566-b3bd-8aab3f91d29c service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:01:53.509315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c3ec15-ee33-4566-b3bd-8aab3f91d29c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:01:53.514136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:01:53.568035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3ec15-ee33-4566-b3bd-8aab3f91d29c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:01:53.569602 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c3ec15-ee33-4566-b3bd-8aab3f91d29c service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:01:53.578783 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1148/2336] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:01:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:02:00.991343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01243e-009c-46d4-a752-451e843391c2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:01.000939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01243e-009c-46d4-a752-451e843391c2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:01.004674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74dbb6-f64c-4edb-a735-6aa952b1bcd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:01.009010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74dbb6-f64c-4edb-a735-6aa952b1bcd9 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:01.009010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74dbb6-f64c-4edb-a735-6aa952b1bcd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:01.009010 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74dbb6-f64c-4edb-a735-6aa952b1bcd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:01.016177 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74dbb6-f64c-4edb-a735-6aa952b1bcd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:01.018180 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74dbb6-f64c-4edb-a735-6aa952b1bcd9 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:01.027931 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1189/2337] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:02:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:02:01.029536 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfeb3-59b6-46df-9450-a854f5eb9f6a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:01.030015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfeb3-59b6-46df-9450-a854f5eb9f6a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:01.030130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7cfeb3-59b6-46df-9450-a854f5eb9f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:01.049511 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-7f7cfeb3-59b6-46df-9450-a854f5eb9f6a None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:01.050897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-7f7cfeb3-59b6-46df-9450-a854f5eb9f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:01.106419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1149/2338] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 20:02:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:02:01.276391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74dbb6-f64c-4edb-a735-6aa952b1bcd9 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:01.277906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2e74dbb6-f64c-4edb-a735-6aa952b1bcd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:01.347271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74dbb6-f64c-4edb-a735-6aa952b1bcd9 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:01.352466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4423fb6b-90cb-4533-b451-0f91c0297357 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:01.353079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423fb6b-90cb-4533-b451-0f91c0297357 service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:01.353383 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423fb6b-90cb-4533-b451-0f91c0297357 service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:01.353638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423fb6b-90cb-4533-b451-0f91c0297357 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:01.362751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4423fb6b-90cb-4533-b451-0f91c0297357 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:01.364629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4423fb6b-90cb-4533-b451-0f91c0297357 service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:01.374034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1190/2339] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 20:02:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 20:02:11.099382 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=50 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:11.370964 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=65 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:19.997983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cfeb3-59b6-46df-9450-a854f5eb9f6a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:19.999412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-7f7cfeb3-59b6-46df-9450-a854f5eb9f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:20.109043 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7cfeb3-59b6-46df-9450-a854f5eb9f6a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:20.115359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32a742-271a-448e-81e5-c299b7d1140d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:20.115359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32a742-271a-448e-81e5-c299b7d1140d service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:20.115359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32a742-271a-448e-81e5-c299b7d1140d service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:20.115359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32a742-271a-448e-81e5-c299b7d1140d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:20.115763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-9c32a742-271a-448e-81e5-c299b7d1140d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:20.188113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32a742-271a-448e-81e5-c299b7d1140d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:20.189541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32a742-271a-448e-81e5-c299b7d1140d service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:20.199420 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1150/2340] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:02:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:02:21.845481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1927fb4-2dc5-495f-a2b9-66bcce4bc013 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:21.846133 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1927fb4-2dc5-495f-a2b9-66bcce4bc013 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:21.846133 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1927fb4-2dc5-495f-a2b9-66bcce4bc013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:21.865715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b1927fb4-2dc5-495f-a2b9-66bcce4bc013 None None] MFA Rules not processed for user `be0920bd9b934f039e5e4de8039346b8`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:21.866954 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b1927fb4-2dc5-495f-a2b9-66bcce4bc013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:21.883059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1191/2341] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:02:21 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:02:26.014435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dda46-dd1b-451f-b652-c379964fde62 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:26.014973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dda46-dd1b-451f-b652-c379964fde62 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:26.015231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dda46-dd1b-451f-b652-c379964fde62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:26.045290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-c45dda46-dd1b-451f-b652-c379964fde62 None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:26.046711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c45dda46-dd1b-451f-b652-c379964fde62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:26.054326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1151/2342] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 20:02:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:02:26.067382 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1927fb4-2dc5-495f-a2b9-66bcce4bc013 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:26.088070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1927fb4-2dc5-495f-a2b9-66bcce4bc013 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:26.093381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba07edc-5391-483b-bb3d-837ce2cad0d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:26.094031 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba07edc-5391-483b-bb3d-837ce2cad0d5 service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:26.094507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba07edc-5391-483b-bb3d-837ce2cad0d5 service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:26.094777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba07edc-5391-483b-bb3d-837ce2cad0d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:26.104943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba07edc-5391-483b-bb3d-837ce2cad0d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:26.107063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba07edc-5391-483b-bb3d-837ce2cad0d5 service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:26.116254 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1192/2343] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 20:02:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 20:02:29.544211 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dda46-dd1b-451f-b652-c379964fde62 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:29.589665 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:29.649526 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dda46-dd1b-451f-b652-c379964fde62 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:29.655783 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262a130-7eef-4211-957c-27bcfe4459e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:29.656481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262a130-7eef-4211-957c-27bcfe4459e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:29.656752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262a130-7eef-4211-957c-27bcfe4459e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:29.657013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262a130-7eef-4211-957c-27bcfe4459e4 admin admin] PATH_INFO: `/v3/users/e7ef98a67a5349ecb7294857daaf203a` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:29.658245 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262a130-7eef-4211-957c-27bcfe4459e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7ef98a67a5349ecb7294857daaf203a)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:29.660861 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262a130-7eef-4211-957c-27bcfe4459e4 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:29.894243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c262a130-7eef-4211-957c-27bcfe4459e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7ef98a67a5349ecb7294857daaf203a'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:29.908435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c262a130-7eef-4211-957c-27bcfe4459e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7ef98a67a5349ecb7294857daaf203a'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:29.991686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c262a130-7eef-4211-957c-27bcfe4459e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7ef98a67a5349ecb7294857daaf203a'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:29.992309 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c262a130-7eef-4211-957c-27bcfe4459e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7ef98a67a5349ecb7294857daaf203a'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:30.000180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c262a130-7eef-4211-957c-27bcfe4459e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7ef98a67a5349ecb7294857daaf203a'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:30.006129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1152/2344] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:02:29 2022] DELETE /identity/v3/users/e7ef98a67a5349ecb7294857daaf203a => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:02:30.010580 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db322416-67a5-4b49-9007-5819e85df89e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:30.011053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db322416-67a5-4b49-9007-5819e85df89e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:30.011150 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db322416-67a5-4b49-9007-5819e85df89e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:30.054188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-db322416-67a5-4b49-9007-5819e85df89e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:30.055428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-db322416-67a5-4b49-9007-5819e85df89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:30.119713 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1193/2345] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:02:30.130616 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262a130-7eef-4211-957c-27bcfe4459e4 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:30.132086 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c262a130-7eef-4211-957c-27bcfe4459e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:30.243185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262a130-7eef-4211-957c-27bcfe4459e4 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:30.249218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4108360-bee3-4d81-97e8-18171af09787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:30.250583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4108360-bee3-4d81-97e8-18171af09787 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:30.250933 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4108360-bee3-4d81-97e8-18171af09787 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:30.251253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4108360-bee3-4d81-97e8-18171af09787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:30.267521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4108360-bee3-4d81-97e8-18171af09787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:30.270656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4108360-bee3-4d81-97e8-18171af09787 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:30.297952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1153/2346] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:02:30.839843 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db322416-67a5-4b49-9007-5819e85df89e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:30.842034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-db322416-67a5-4b49-9007-5819e85df89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:30.856210 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db322416-67a5-4b49-9007-5819e85df89e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:30.861904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:30.862555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:30.862833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:30.863108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] PATH_INFO: `/v3/projects/9d68672014624b05a7115633aacea896` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:30.864247 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d68672014624b05a7115633aacea896)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:30.866834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:31.034035 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d68672014624b05a7115633aacea896', 'request_id': 'req-d248ec22-728b-43c8-8b0b-6061ddcb8c58'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:31.140839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d68672014624b05a7115633aacea896', 'request_id': 'req-d248ec22-728b-43c8-8b0b-6061ddcb8c58'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:31.147901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d68672014624b05a7115633aacea896', 'request_id': 'req-d248ec22-728b-43c8-8b0b-6061ddcb8c58'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:31.159495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] The token cache is being invalidate because project 9d68672014624b05a7115633aacea896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:02:31.159801 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:31.163546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:31.163546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1194/2347] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:02:30 2022] DELETE /identity/v3/projects/9d68672014624b05a7115633aacea896 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:02:31.166034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4108360-bee3-4d81-97e8-18171af09787 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:31.175751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4108360-bee3-4d81-97e8-18171af09787 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:31.215983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:31.232934 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1430a484-b0cf-4628-9cca-df9eafdae0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:31.233596 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430a484-b0cf-4628-9cca-df9eafdae0f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:31.233848 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430a484-b0cf-4628-9cca-df9eafdae0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:31.234122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1430a484-b0cf-4628-9cca-df9eafdae0f7 admin admin] PATH_INFO: `/v3/users/21a11bc000dc4bd4b28316421e5f3cb8` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:31.235243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430a484-b0cf-4628-9cca-df9eafdae0f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21a11bc000dc4bd4b28316421e5f3cb8)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:31.237154 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1430a484-b0cf-4628-9cca-df9eafdae0f7 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:31.549570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1430a484-b0cf-4628-9cca-df9eafdae0f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21a11bc000dc4bd4b28316421e5f3cb8'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:31.562167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1430a484-b0cf-4628-9cca-df9eafdae0f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21a11bc000dc4bd4b28316421e5f3cb8'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:31.643467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1430a484-b0cf-4628-9cca-df9eafdae0f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21a11bc000dc4bd4b28316421e5f3cb8'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:31.644103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1430a484-b0cf-4628-9cca-df9eafdae0f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21a11bc000dc4bd4b28316421e5f3cb8'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:31.652158 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-1430a484-b0cf-4628-9cca-df9eafdae0f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21a11bc000dc4bd4b28316421e5f3cb8'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:31.659104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1154/2348] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:02:31 2022] DELETE /identity/v3/users/21a11bc000dc4bd4b28316421e5f3cb8 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:02:32.150648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:32.152862 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:32.166992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248ec22-728b-43c8-8b0b-6061ddcb8c58 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:32.207572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:32.228877 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:32.229522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:32.230034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:32.230510 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27 admin admin] PATH_INFO: `/v3/projects/d8a63a6079774d9eac59bb74ad415a12` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:32.231727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8a63a6079774d9eac59bb74ad415a12)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:32.234398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:32.341063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8a63a6079774d9eac59bb74ad415a12', 'request_id': 'req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:32.439664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8a63a6079774d9eac59bb74ad415a12', 'request_id': 'req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:32.449310 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8a63a6079774d9eac59bb74ad415a12', 'request_id': 'req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:32.462358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27 admin admin] The token cache is being invalidate because project d8a63a6079774d9eac59bb74ad415a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:02:32.462646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:32.463324 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6c1f7c2c-8d3c-46e0-9fa1-5c2011851b27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:32.465067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1195/2349] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:02:32 2022] DELETE /identity/v3/projects/d8a63a6079774d9eac59bb74ad415a12 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:02:32.471379 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75a97c-652b-40ad-afec-afa01864d5b3 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:32.471873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75a97c-652b-40ad-afec-afa01864d5b3 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:32.472111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75a97c-652b-40ad-afec-afa01864d5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:32.535543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-eb75a97c-652b-40ad-afec-afa01864d5b3 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:32.536748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-eb75a97c-652b-40ad-afec-afa01864d5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:32.595323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1155/2350] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:02:32.603091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e71bf-2771-4966-b969-331bd6a05379 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:32.603402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e71bf-2771-4966-b969-331bd6a05379 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:32.603677 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7e71bf-2771-4966-b969-331bd6a05379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:32.626817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-be7e71bf-2771-4966-b969-331bd6a05379 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:32.628737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-be7e71bf-2771-4966-b969-331bd6a05379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:32.638725 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1196/2351] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:02:32.642985 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75a97c-652b-40ad-afec-afa01864d5b3 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:32.653327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75a97c-652b-40ad-afec-afa01864d5b3 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:32.668258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efb370-f747-4698-a477-b1cc1a82c7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:32.668258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efb370-f747-4698-a477-b1cc1a82c7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:32.668258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efb370-f747-4698-a477-b1cc1a82c7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:32.668258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2efb370-f747-4698-a477-b1cc1a82c7e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:32.668584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efb370-f747-4698-a477-b1cc1a82c7e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:02:32.668584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efb370-f747-4698-a477-b1cc1a82c7e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:32.668584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2efb370-f747-4698-a477-b1cc1a82c7e8 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:32.676502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1156/2352] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 20:02:32.680946 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e71bf-2771-4966-b969-331bd6a05379 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:32.693260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7e71bf-2771-4966-b969-331bd6a05379 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:32.705894 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8eb6b-6516-4408-9e62-a927bc581273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:32.706628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8eb6b-6516-4408-9e62-a927bc581273 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:32.706909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8eb6b-6516-4408-9e62-a927bc581273 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:32.707174 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8eb6b-6516-4408-9e62-a927bc581273 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:32.707799 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8eb6b-6516-4408-9e62-a927bc581273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:32.709971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8eb6b-6516-4408-9e62-a927bc581273 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:32.813098 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1197/2353] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:02:32 2022] POST /identity/v3/projects => generated 409 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:02:32.817196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efb370-f747-4698-a477-b1cc1a82c7e8 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:32.831631 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2efb370-f747-4698-a477-b1cc1a82c7e8 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:32.875520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:32.915210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082df06d-d511-40a4-a7bc-dc723330954c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:32.915847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082df06d-d511-40a4-a7bc-dc723330954c admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:32.916129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082df06d-d511-40a4-a7bc-dc723330954c admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:32.916460 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082df06d-d511-40a4-a7bc-dc723330954c admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:32.917114 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082df06d-d511-40a4-a7bc-dc723330954c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:32.919098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082df06d-d511-40a4-a7bc-dc723330954c admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:33.004298 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1157/2354] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:02:32 2022] POST /identity/v3/users => generated 305 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:02:33.008292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8eb6b-6516-4408-9e62-a927bc581273 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:33.028513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8eb6b-6516-4408-9e62-a927bc581273 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:33.034431 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2813822-ae10-4eea-b863-d1d52da103ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:33.035140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2813822-ae10-4eea-b863-d1d52da103ad admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:33.035422 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2813822-ae10-4eea-b863-d1d52da103ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:33.035776 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2813822-ae10-4eea-b863-d1d52da103ad admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:33.036498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2813822-ae10-4eea-b863-d1d52da103ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:33.038391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2813822-ae10-4eea-b863-d1d52da103ad admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:33.067321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1198/2355] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:02:33.071211 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082df06d-d511-40a4-a7bc-dc723330954c admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:33.084971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082df06d-d511-40a4-a7bc-dc723330954c admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:33.090506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c063f3-0f17-4090-af79-a80bf63eb5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:33.091989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c063f3-0f17-4090-af79-a80bf63eb5db admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:33.091989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c063f3-0f17-4090-af79-a80bf63eb5db admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:33.092178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c063f3-0f17-4090-af79-a80bf63eb5db admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:33.093206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c063f3-0f17-4090-af79-a80bf63eb5db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:33.095325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c063f3-0f17-4090-af79-a80bf63eb5db admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:33.110756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1158/2356] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:02:33.111855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2813822-ae10-4eea-b863-d1d52da103ad admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:33.122115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2813822-ae10-4eea-b863-d1d52da103ad admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:33.127316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43260d3-aa27-40f4-8254-6c0197d47896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:33.128011 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43260d3-aa27-40f4-8254-6c0197d47896 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:33.128285 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43260d3-aa27-40f4-8254-6c0197d47896 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:33.128556 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43260d3-aa27-40f4-8254-6c0197d47896 admin admin] PATH_INFO: `/v3/projects/84f081c7cb694198a477f412e09eed7f/users/cb3f71f493a846a4a0eb62e32157eafc/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:33.144632 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43260d3-aa27-40f4-8254-6c0197d47896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f081c7cb694198a477f412e09eed7f, user_id=cb3f71f493a846a4a0eb62e32157eafc, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:33.147327 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43260d3-aa27-40f4-8254-6c0197d47896 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:33.267279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1199/2357] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:02:33 2022] PUT /identity/v3/projects/84f081c7cb694198a477f412e09eed7f/users/cb3f71f493a846a4a0eb62e32157eafc/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:02:33.277830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1ff8f-dd29-402d-87ce-14de5407f471 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:33.278168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1ff8f-dd29-402d-87ce-14de5407f471 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:33.278435 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1ff8f-dd29-402d-87ce-14de5407f471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:33.307107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-04d1ff8f-dd29-402d-87ce-14de5407f471 None None] MFA Rules not processed for user `cb3f71f493a846a4a0eb62e32157eafc`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:33.311590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-04d1ff8f-dd29-402d-87ce-14de5407f471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:33.381744 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1159/2358] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:02:33 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:02:33.386073 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43260d3-aa27-40f4-8254-6c0197d47896 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:33.451126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:33.454790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43260d3-aa27-40f4-8254-6c0197d47896 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:33.459708 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bb494-5574-436c-99d8-9cbe03ad9247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:33.460309 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bb494-5574-436c-99d8-9cbe03ad9247 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:33.460583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bb494-5574-436c-99d8-9cbe03ad9247 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:33.460919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bb494-5574-436c-99d8-9cbe03ad9247 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:33.461470 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bb494-5574-436c-99d8-9cbe03ad9247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:33.463415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bb494-5574-436c-99d8-9cbe03ad9247 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:33.579996 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1200/2359] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:02:33 2022] POST /identity/v3/projects => generated 407 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:02:33.581254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d1ff8f-dd29-402d-87ce-14de5407f471 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:33.593090 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d1ff8f-dd29-402d-87ce-14de5407f471 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:33.648268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d5966-0249-4a75-9679-01ad264cc92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:33.648872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d5966-0249-4a75-9679-01ad264cc92d admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:33.649149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d5966-0249-4a75-9679-01ad264cc92d admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:33.649420 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d5966-0249-4a75-9679-01ad264cc92d admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:33.650157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d5966-0249-4a75-9679-01ad264cc92d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:33.652086 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83d5966-0249-4a75-9679-01ad264cc92d admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:33.758710 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1160/2360] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:02:33 2022] POST /identity/v3/users => generated 310 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:02:33.762968 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bb494-5574-436c-99d8-9cbe03ad9247 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:33.773818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bb494-5574-436c-99d8-9cbe03ad9247 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:33.778657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96087b-7af7-4f23-b07f-b426c2c6b438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:33.779916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96087b-7af7-4f23-b07f-b426c2c6b438 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:33.780192 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96087b-7af7-4f23-b07f-b426c2c6b438 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:33.780462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96087b-7af7-4f23-b07f-b426c2c6b438 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:33.781125 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96087b-7af7-4f23-b07f-b426c2c6b438 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:33.782851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96087b-7af7-4f23-b07f-b426c2c6b438 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:33.792509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1201/2361] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:02:33.800533 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d5966-0249-4a75-9679-01ad264cc92d admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:33.810516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d5966-0249-4a75-9679-01ad264cc92d admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:33.815260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50591c-8479-4d85-9ef5-105c3b87e47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:33.815898 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50591c-8479-4d85-9ef5-105c3b87e47e admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:33.816174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50591c-8479-4d85-9ef5-105c3b87e47e admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:33.816443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50591c-8479-4d85-9ef5-105c3b87e47e admin admin] PATH_INFO: `/v3/projects/e2baa59f17c5459f9677e1f9b04950d4/users/4940b5c87d6747dd8d06387ac3b355b0/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:33.833305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50591c-8479-4d85-9ef5-105c3b87e47e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2baa59f17c5459f9677e1f9b04950d4, user_id=4940b5c87d6747dd8d06387ac3b355b0, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:33.836338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50591c-8479-4d85-9ef5-105c3b87e47e admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:33.907594 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:33.916008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1161/2362] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:02:33 2022] PUT /identity/v3/projects/e2baa59f17c5459f9677e1f9b04950d4/users/4940b5c87d6747dd8d06387ac3b355b0/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:02:33.918351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96087b-7af7-4f23-b07f-b426c2c6b438 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:33.929317 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96087b-7af7-4f23-b07f-b426c2c6b438 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:33.990314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52df16bc-9fcc-483e-aa55-d246c359c372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:33.991046 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df16bc-9fcc-483e-aa55-d246c359c372 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:33.991357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df16bc-9fcc-483e-aa55-d246c359c372 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:33.991593 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52df16bc-9fcc-483e-aa55-d246c359c372 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:33.992397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df16bc-9fcc-483e-aa55-d246c359c372 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:33.994149 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52df16bc-9fcc-483e-aa55-d246c359c372 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:34.005684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1202/2363] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:02:34.012142 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50591c-8479-4d85-9ef5-105c3b87e47e admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:34.021395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50591c-8479-4d85-9ef5-105c3b87e47e admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:34.026192 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f994f1-04e0-4394-9444-0842ffd89f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:34.026968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f994f1-04e0-4394-9444-0842ffd89f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:34.027247 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f994f1-04e0-4394-9444-0842ffd89f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:34.027520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f994f1-04e0-4394-9444-0842ffd89f6d admin admin] PATH_INFO: `/v3/projects/e2baa59f17c5459f9677e1f9b04950d4/users/4940b5c87d6747dd8d06387ac3b355b0/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:34.031076 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f994f1-04e0-4394-9444-0842ffd89f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2baa59f17c5459f9677e1f9b04950d4, user_id=4940b5c87d6747dd8d06387ac3b355b0, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:34.035078 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f994f1-04e0-4394-9444-0842ffd89f6d admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:34.056846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1162/2364] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:02:34 2022] PUT /identity/v3/projects/e2baa59f17c5459f9677e1f9b04950d4/users/4940b5c87d6747dd8d06387ac3b355b0/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:02:34.067881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6670f1-0dd4-419f-839f-bf63fbc080c4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:34.068166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6670f1-0dd4-419f-839f-bf63fbc080c4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:34.068515 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6670f1-0dd4-419f-839f-bf63fbc080c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:34.104922 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0c6670f1-0dd4-419f-839f-bf63fbc080c4 None None] MFA Rules not processed for user `4940b5c87d6747dd8d06387ac3b355b0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:34.106486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0c6670f1-0dd4-419f-839f-bf63fbc080c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:34.176780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1203/2365] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:02:34 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:02:34.187666 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f994f1-04e0-4394-9444-0842ffd89f6d admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:34.189546 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d1f994f1-04e0-4394-9444-0842ffd89f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:34.293748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f994f1-04e0-4394-9444-0842ffd89f6d admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:34.299772 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcf0265-4d1e-4208-a57c-a5ddc80d52fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:34.300704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf0265-4d1e-4208-a57c-a5ddc80d52fd service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:34.301567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf0265-4d1e-4208-a57c-a5ddc80d52fd service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:34.302086 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf0265-4d1e-4208-a57c-a5ddc80d52fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:34.317581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf0265-4d1e-4208-a57c-a5ddc80d52fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:34.320206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcf0265-4d1e-4208-a57c-a5ddc80d52fd service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:34.335160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1163/2366] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:02:34 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:02:34.795123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6670f1-0dd4-419f-839f-bf63fbc080c4 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:34.808118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6670f1-0dd4-419f-839f-bf63fbc080c4 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:34.813353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e90495-d7c5-4192-83a7-3eda09ab0161 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:34.814115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e90495-d7c5-4192-83a7-3eda09ab0161 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:34.814464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e90495-d7c5-4192-83a7-3eda09ab0161 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:34.814852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e90495-d7c5-4192-83a7-3eda09ab0161 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:34.887275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:34.901252 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e90495-d7c5-4192-83a7-3eda09ab0161 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:34.903952 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e90495-d7c5-4192-83a7-3eda09ab0161 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:34.914985 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1204/2367] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:02:34 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:02:44.327831 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:44.963554 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:46.409783 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901dc49-29d3-40f4-8a42-c351be4fc16a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:46.410299 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901dc49-29d3-40f4-8a42-c351be4fc16a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:46.410421 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e901dc49-29d3-40f4-8a42-c351be4fc16a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:46.434669 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e901dc49-29d3-40f4-8a42-c351be4fc16a None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:46.436451 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e901dc49-29d3-40f4-8a42-c351be4fc16a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:46.492953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1164/2368] 10.222.0.42 () {58 vars in 967 bytes} [Wed Sep 14 20:02:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:02:46.506003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e90495-d7c5-4192-83a7-3eda09ab0161 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:46.507433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-60e90495-d7c5-4192-83a7-3eda09ab0161 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:46.578389 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e90495-d7c5-4192-83a7-3eda09ab0161 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:46.583576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63346ca1-92db-4766-983d-f358bb974854 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:46.584175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63346ca1-92db-4766-983d-f358bb974854 service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:46.584745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63346ca1-92db-4766-983d-f358bb974854 service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:46.585030 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63346ca1-92db-4766-983d-f358bb974854 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:46.593272 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63346ca1-92db-4766-983d-f358bb974854 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:46.594989 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63346ca1-92db-4766-983d-f358bb974854 service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:46.603200 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1205/2369] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 20:02:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 20:02:46.700344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901dc49-29d3-40f4-8a42-c351be4fc16a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:46.764598 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e901dc49-29d3-40f4-8a42-c351be4fc16a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:46.769021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb30835-ed18-490f-848f-2408fffe92f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:46.770410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb30835-ed18-490f-848f-2408fffe92f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:46.770410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb30835-ed18-490f-848f-2408fffe92f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:46.770410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb30835-ed18-490f-848f-2408fffe92f5 admin admin] PATH_INFO: `/v3/users/cb3f71f493a846a4a0eb62e32157eafc` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:46.771402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb30835-ed18-490f-848f-2408fffe92f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb3f71f493a846a4a0eb62e32157eafc)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:46.773436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb30835-ed18-490f-848f-2408fffe92f5 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:47.003187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3cb30835-ed18-490f-848f-2408fffe92f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb3f71f493a846a4a0eb62e32157eafc'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:47.015602 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3cb30835-ed18-490f-848f-2408fffe92f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb3f71f493a846a4a0eb62e32157eafc'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:47.088512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3cb30835-ed18-490f-848f-2408fffe92f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb3f71f493a846a4a0eb62e32157eafc'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:47.089147 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3cb30835-ed18-490f-848f-2408fffe92f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb3f71f493a846a4a0eb62e32157eafc'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:47.097461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3cb30835-ed18-490f-848f-2408fffe92f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb3f71f493a846a4a0eb62e32157eafc'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:47.103673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1165/2370] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:02:46 2022] DELETE /identity/v3/users/cb3f71f493a846a4a0eb62e32157eafc => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:02:47.108932 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e07dd-6475-4e2c-b998-177683d356da None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:47.109222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e07dd-6475-4e2c-b998-177683d356da None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:47.109492 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e07dd-6475-4e2c-b998-177683d356da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:47.128621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8e5e07dd-6475-4e2c-b998-177683d356da None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:47.129855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8e5e07dd-6475-4e2c-b998-177683d356da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:47.191043 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1206/2371] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:02:47.200958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb30835-ed18-490f-848f-2408fffe92f5 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:47.202599 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-3cb30835-ed18-490f-848f-2408fffe92f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:47.235055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:47.268219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb30835-ed18-490f-848f-2408fffe92f5 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:47.275156 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb0cd9-96a9-4938-a3a3-86f05ea1e817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:47.275836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb0cd9-96a9-4938-a3a3-86f05ea1e817 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:47.276320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb0cd9-96a9-4938-a3a3-86f05ea1e817 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:47.276601 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb0cd9-96a9-4938-a3a3-86f05ea1e817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:47.288358 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb0cd9-96a9-4938-a3a3-86f05ea1e817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:47.290026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29eb0cd9-96a9-4938-a3a3-86f05ea1e817 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:47.306260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1166/2372] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:02:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:02:48.090716 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e07dd-6475-4e2c-b998-177683d356da None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:48.090716 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8e5e07dd-6475-4e2c-b998-177683d356da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:48.090716 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e07dd-6475-4e2c-b998-177683d356da None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:48.092616 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:48.092616 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:48.092616 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:48.092616 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] PATH_INFO: `/v3/projects/84f081c7cb694198a477f412e09eed7f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:48.093432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f081c7cb694198a477f412e09eed7f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:48.093432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:48.210210 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f081c7cb694198a477f412e09eed7f', 'request_id': 'req-f3aea558-abe5-49e5-a32e-171952dbb147'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:48.294654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f081c7cb694198a477f412e09eed7f', 'request_id': 'req-f3aea558-abe5-49e5-a32e-171952dbb147'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:48.302193 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f081c7cb694198a477f412e09eed7f', 'request_id': 'req-f3aea558-abe5-49e5-a32e-171952dbb147'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:48.313341 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] The token cache is being invalidate because project 84f081c7cb694198a477f412e09eed7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:02:48.313621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:48.314748 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:48.316264 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1207/2373] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:02:48 2022] DELETE /identity/v3/projects/84f081c7cb694198a477f412e09eed7f => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:02:48.320194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb0cd9-96a9-4938-a3a3-86f05ea1e817 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:48.330089 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb0cd9-96a9-4938-a3a3-86f05ea1e817 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:48.391226 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76139f53-bdfb-4bdc-b5fc-7864708787c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:48.392277 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76139f53-bdfb-4bdc-b5fc-7864708787c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:48.392734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76139f53-bdfb-4bdc-b5fc-7864708787c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:48.393173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76139f53-bdfb-4bdc-b5fc-7864708787c4 admin admin] PATH_INFO: `/v3/users/4940b5c87d6747dd8d06387ac3b355b0` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:48.394597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76139f53-bdfb-4bdc-b5fc-7864708787c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4940b5c87d6747dd8d06387ac3b355b0)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:48.396611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76139f53-bdfb-4bdc-b5fc-7864708787c4 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:48.675183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-76139f53-bdfb-4bdc-b5fc-7864708787c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4940b5c87d6747dd8d06387ac3b355b0'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:48.686923 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-76139f53-bdfb-4bdc-b5fc-7864708787c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4940b5c87d6747dd8d06387ac3b355b0'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:48.775162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-76139f53-bdfb-4bdc-b5fc-7864708787c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4940b5c87d6747dd8d06387ac3b355b0'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:48.775854 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-76139f53-bdfb-4bdc-b5fc-7864708787c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4940b5c87d6747dd8d06387ac3b355b0'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:48.784066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-76139f53-bdfb-4bdc-b5fc-7864708787c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4940b5c87d6747dd8d06387ac3b355b0'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:48.790100 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1167/2374] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:02:48 2022] DELETE /identity/v3/users/4940b5c87d6747dd8d06387ac3b355b0 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:02:49.330895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:49.334809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:49.347165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aea558-abe5-49e5-a32e-171952dbb147 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:49.367408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:49.414085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:49.414723 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:49.415004 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:49.415275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006 admin admin] PATH_INFO: `/v3/projects/e2baa59f17c5459f9677e1f9b04950d4` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:49.416386 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2baa59f17c5459f9677e1f9b04950d4)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:49.418385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:49.586476 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2baa59f17c5459f9677e1f9b04950d4', 'request_id': 'req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:49.723584 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2baa59f17c5459f9677e1f9b04950d4', 'request_id': 'req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:49.730918 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2baa59f17c5459f9677e1f9b04950d4', 'request_id': 'req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:49.742779 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006 admin admin] The token cache is being invalidate because project e2baa59f17c5459f9677e1f9b04950d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:02:49.743144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:49.743871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-e77ceaa3-34ec-4670-8ad2-453a8e1bf006 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:02:49.745439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1208/2375] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:02:49 2022] DELETE /identity/v3/projects/e2baa59f17c5459f9677e1f9b04950d4 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:02:49.750338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c741d-c444-4a75-b000-e30527b85626 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:49.750846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c741d-c444-4a75-b000-e30527b85626 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:49.750956 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c741d-c444-4a75-b000-e30527b85626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:49.774704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-742c741d-c444-4a75-b000-e30527b85626 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:49.776021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-742c741d-c444-4a75-b000-e30527b85626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:49.831494 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:49.858153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1168/2376] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:02:49.863416 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d7121-e423-4388-8a0e-0806b871aa8d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:49.863918 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d7121-e423-4388-8a0e-0806b871aa8d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:49.864268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d7121-e423-4388-8a0e-0806b871aa8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:49.891189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-384d7121-e423-4388-8a0e-0806b871aa8d None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:49.892446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-384d7121-e423-4388-8a0e-0806b871aa8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:49.901680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1209/2377] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:02:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:02:49.907774 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c741d-c444-4a75-b000-e30527b85626 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:49.919578 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c741d-c444-4a75-b000-e30527b85626 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:49.927250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f0d04-2d6d-4664-b91d-ad227883e107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:49.927250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f0d04-2d6d-4664-b91d-ad227883e107 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:49.927250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f0d04-2d6d-4664-b91d-ad227883e107 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:49.927250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f0d04-2d6d-4664-b91d-ad227883e107 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:49.927809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f0d04-2d6d-4664-b91d-ad227883e107 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:02:49.927809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f0d04-2d6d-4664-b91d-ad227883e107 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:49.928704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f0d04-2d6d-4664-b91d-ad227883e107 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:49.944012 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1169/2378] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:02:49.949294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d7121-e423-4388-8a0e-0806b871aa8d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:49.960471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d7121-e423-4388-8a0e-0806b871aa8d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:49.965740 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70f125-4852-4294-98c0-59a3bd7de1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:49.966326 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70f125-4852-4294-98c0-59a3bd7de1a9 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:49.966637 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70f125-4852-4294-98c0-59a3bd7de1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:49.966948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70f125-4852-4294-98c0-59a3bd7de1a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:49.968265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70f125-4852-4294-98c0-59a3bd7de1a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:49.970223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70f125-4852-4294-98c0-59a3bd7de1a9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:50.147274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1210/2379] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:02:49 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:02:50.148452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f0d04-2d6d-4664-b91d-ad227883e107 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:50.159986 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f0d04-2d6d-4664-b91d-ad227883e107 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:50.247130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6429395-1dc3-4679-87bb-baf3920545bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:50.247809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6429395-1dc3-4679-87bb-baf3920545bf admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:50.248268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6429395-1dc3-4679-87bb-baf3920545bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:50.249251 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6429395-1dc3-4679-87bb-baf3920545bf admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:50.250021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6429395-1dc3-4679-87bb-baf3920545bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:50.252192 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6429395-1dc3-4679-87bb-baf3920545bf admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:50.341921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1170/2380] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:02:50 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:02:50.348374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70f125-4852-4294-98c0-59a3bd7de1a9 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:50.359898 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70f125-4852-4294-98c0-59a3bd7de1a9 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:50.365121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a9e86-0c63-42ab-b8a5-96ea23b791bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:50.365749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a9e86-0c63-42ab-b8a5-96ea23b791bc admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:50.366317 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a9e86-0c63-42ab-b8a5-96ea23b791bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:50.366594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a9e86-0c63-42ab-b8a5-96ea23b791bc admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:50.367226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a9e86-0c63-42ab-b8a5-96ea23b791bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:50.368945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a9e86-0c63-42ab-b8a5-96ea23b791bc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:50.385408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1211/2381] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:02:50.388550 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6429395-1dc3-4679-87bb-baf3920545bf admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:50.400267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6429395-1dc3-4679-87bb-baf3920545bf admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:50.406042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bff31-6aaf-4dec-9bac-ff9a788bec1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:50.406874 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bff31-6aaf-4dec-9bac-ff9a788bec1c admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:50.407301 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bff31-6aaf-4dec-9bac-ff9a788bec1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:50.407847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bff31-6aaf-4dec-9bac-ff9a788bec1c admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:50.408777 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bff31-6aaf-4dec-9bac-ff9a788bec1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:50.410855 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bff31-6aaf-4dec-9bac-ff9a788bec1c admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:50.427859 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1171/2382] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:02:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:02:50.429128 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a9e86-0c63-42ab-b8a5-96ea23b791bc admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:50.442605 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a9e86-0c63-42ab-b8a5-96ea23b791bc admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:50.448384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c2888-91fc-4c93-b251-afd73d4f88f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:50.454414 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c2888-91fc-4c93-b251-afd73d4f88f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:50.454414 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c2888-91fc-4c93-b251-afd73d4f88f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:50.454414 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c2888-91fc-4c93-b251-afd73d4f88f9 admin admin] PATH_INFO: `/v3/projects/7cfdb98bde394ef390d0dea491dd8a1e/users/edc06cf22707473eb7c0b3a343b892c7/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:50.467337 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c2888-91fc-4c93-b251-afd73d4f88f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cfdb98bde394ef390d0dea491dd8a1e, user_id=edc06cf22707473eb7c0b3a343b892c7, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:50.470223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c2888-91fc-4c93-b251-afd73d4f88f9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:50.569694 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1212/2383] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:02:50 2022] PUT /identity/v3/projects/7cfdb98bde394ef390d0dea491dd8a1e/users/edc06cf22707473eb7c0b3a343b892c7/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:02:50.581945 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00468bcf-032c-401f-b3b3-f86a3aac84ad None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:50.582236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00468bcf-032c-401f-b3b3-f86a3aac84ad None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:50.582511 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00468bcf-032c-401f-b3b3-f86a3aac84ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:50.613719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-00468bcf-032c-401f-b3b3-f86a3aac84ad None None] MFA Rules not processed for user `edc06cf22707473eb7c0b3a343b892c7`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:50.615012 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-00468bcf-032c-401f-b3b3-f86a3aac84ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:50.681312 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1172/2384] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:02:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:02:50.686919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c2888-91fc-4c93-b251-afd73d4f88f9 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:50.729651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:50.748221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c2888-91fc-4c93-b251-afd73d4f88f9 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:50.753215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d6cb4-47ea-4563-a5e1-e37abfb3b54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:50.754521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d6cb4-47ea-4563-a5e1-e37abfb3b54c admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:50.754521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d6cb4-47ea-4563-a5e1-e37abfb3b54c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:50.754521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d6cb4-47ea-4563-a5e1-e37abfb3b54c admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:50.755020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d6cb4-47ea-4563-a5e1-e37abfb3b54c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:50.757117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d6cb4-47ea-4563-a5e1-e37abfb3b54c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:50.868158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1213/2385] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:02:50 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:02:50.872260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00468bcf-032c-401f-b3b3-f86a3aac84ad None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:50.912563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00468bcf-032c-401f-b3b3-f86a3aac84ad None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:50.959139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:50.976130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91d41d-a373-4679-ae85-ef58ea570c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:50.976801 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91d41d-a373-4679-ae85-ef58ea570c22 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:50.977081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91d41d-a373-4679-ae85-ef58ea570c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:50.977469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91d41d-a373-4679-ae85-ef58ea570c22 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:50.978200 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91d41d-a373-4679-ae85-ef58ea570c22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:50.980282 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91d41d-a373-4679-ae85-ef58ea570c22 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:51.087635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1173/2386] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:02:50 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:02:51.088921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d6cb4-47ea-4563-a5e1-e37abfb3b54c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:51.099286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d6cb4-47ea-4563-a5e1-e37abfb3b54c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:51.103875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d02c20-61c2-489a-9267-f3c913eff8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:51.104467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d02c20-61c2-489a-9267-f3c913eff8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:51.104786 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d02c20-61c2-489a-9267-f3c913eff8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:51.105057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d02c20-61c2-489a-9267-f3c913eff8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:51.105684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d02c20-61c2-489a-9267-f3c913eff8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:51.107301 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d02c20-61c2-489a-9267-f3c913eff8f5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:51.117371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1214/2387] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:02:51.121508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91d41d-a373-4679-ae85-ef58ea570c22 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:51.132133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91d41d-a373-4679-ae85-ef58ea570c22 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:51.136870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3925f64-e360-4706-addd-b156165f8bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:51.138231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3925f64-e360-4706-addd-b156165f8bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:51.138231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3925f64-e360-4706-addd-b156165f8bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:51.138231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3925f64-e360-4706-addd-b156165f8bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:51.138703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3925f64-e360-4706-addd-b156165f8bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:51.140879 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3925f64-e360-4706-addd-b156165f8bc2 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:51.150706 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1174/2388] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:02:51.154804 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d02c20-61c2-489a-9267-f3c913eff8f5 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:51.167270 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d02c20-61c2-489a-9267-f3c913eff8f5 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:51.183582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c5f13-71c8-4e00-b474-ed06aa2b212c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:51.184373 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c5f13-71c8-4e00-b474-ed06aa2b212c admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:51.184750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c5f13-71c8-4e00-b474-ed06aa2b212c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:51.185033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c5f13-71c8-4e00-b474-ed06aa2b212c admin admin] PATH_INFO: `/v3/projects/e6e3f0fb2ffb48229c57f83e6018dac2/users/2f88405b171f42e7a4b63c217129244f/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:51.204856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c5f13-71c8-4e00-b474-ed06aa2b212c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6e3f0fb2ffb48229c57f83e6018dac2, user_id=2f88405b171f42e7a4b63c217129244f, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:51.205534 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c5f13-71c8-4e00-b474-ed06aa2b212c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:51.314105 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1215/2389] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:02:51 2022] PUT /identity/v3/projects/e6e3f0fb2ffb48229c57f83e6018dac2/users/2f88405b171f42e7a4b63c217129244f/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:02:51.324200 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b48b7-44cc-4a9f-892e-544bec14b731 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:51.324485 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b48b7-44cc-4a9f-892e-544bec14b731 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:51.324754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b48b7-44cc-4a9f-892e-544bec14b731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:51.355451 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b08b48b7-44cc-4a9f-892e-544bec14b731 None None] MFA Rules not processed for user `2f88405b171f42e7a4b63c217129244f`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:51.356712 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b08b48b7-44cc-4a9f-892e-544bec14b731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:51.431394 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1175/2390] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:02:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:02:51.436086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c5f13-71c8-4e00-b474-ed06aa2b212c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:51.514350 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c5f13-71c8-4e00-b474-ed06aa2b212c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:51.520737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d17466-3b4d-43dd-9a11-830b40ead463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:51.522228 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d17466-3b4d-43dd-9a11-830b40ead463 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:51.523109 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d17466-3b4d-43dd-9a11-830b40ead463 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:51.523472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d17466-3b4d-43dd-9a11-830b40ead463 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:51.524138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d17466-3b4d-43dd-9a11-830b40ead463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:51.527329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d17466-3b4d-43dd-9a11-830b40ead463 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:51.626578 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1216/2391] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:02:51 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:02:51.631606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b48b7-44cc-4a9f-892e-544bec14b731 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:51.643524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b48b7-44cc-4a9f-892e-544bec14b731 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:51.705604 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d0fe9-195c-47e1-a1f9-d6ece829d172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:51.710507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d0fe9-195c-47e1-a1f9-d6ece829d172 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:51.710507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d0fe9-195c-47e1-a1f9-d6ece829d172 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:51.710507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47d0fe9-195c-47e1-a1f9-d6ece829d172 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:51.710507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d0fe9-195c-47e1-a1f9-d6ece829d172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:51.710507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47d0fe9-195c-47e1-a1f9-d6ece829d172 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:51.839129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1176/2392] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:02:51 2022] POST /identity/v3/users => generated 299 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:02:51.842768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d17466-3b4d-43dd-9a11-830b40ead463 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:51.853382 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d17466-3b4d-43dd-9a11-830b40ead463 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:51.858542 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62630b55-a5f7-49f0-8ffb-b6e28668e1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:51.859161 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62630b55-a5f7-49f0-8ffb-b6e28668e1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:51.859497 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62630b55-a5f7-49f0-8ffb-b6e28668e1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:51.859771 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62630b55-a5f7-49f0-8ffb-b6e28668e1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:51.860403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62630b55-a5f7-49f0-8ffb-b6e28668e1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:51.862083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62630b55-a5f7-49f0-8ffb-b6e28668e1d1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:51.877651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1217/2393] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:02:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:02:51.878852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d0fe9-195c-47e1-a1f9-d6ece829d172 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:51.895952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47d0fe9-195c-47e1-a1f9-d6ece829d172 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:51.895952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29436095-5f83-487e-a636-8dd92670dfa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:51.895952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29436095-5f83-487e-a636-8dd92670dfa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:51.895952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29436095-5f83-487e-a636-8dd92670dfa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:51.896366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29436095-5f83-487e-a636-8dd92670dfa7 admin admin] PATH_INFO: `/v3/projects/12350c50b0674c93861cb9250b9738a7/users/08365578ac624d3e8ae7a683b2c3b713/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:51.911224 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29436095-5f83-487e-a636-8dd92670dfa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12350c50b0674c93861cb9250b9738a7, user_id=08365578ac624d3e8ae7a683b2c3b713, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:51.914001 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29436095-5f83-487e-a636-8dd92670dfa7 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:52.026823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1177/2394] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:02:51 2022] PUT /identity/v3/projects/12350c50b0674c93861cb9250b9738a7/users/08365578ac624d3e8ae7a683b2c3b713/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:02:52.031308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62630b55-a5f7-49f0-8ffb-b6e28668e1d1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:52.041507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62630b55-a5f7-49f0-8ffb-b6e28668e1d1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:52.112182 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f65a67d-483a-481b-9b7f-448fc2773a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:52.112805 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65a67d-483a-481b-9b7f-448fc2773a66 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:52.113080 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65a67d-483a-481b-9b7f-448fc2773a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:52.113347 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f65a67d-483a-481b-9b7f-448fc2773a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:52.114014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65a67d-483a-481b-9b7f-448fc2773a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:52.115614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f65a67d-483a-481b-9b7f-448fc2773a66 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:52.121270 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:52.130370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1218/2395] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:02:52.131587 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29436095-5f83-487e-a636-8dd92670dfa7 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:52.143778 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29436095-5f83-487e-a636-8dd92670dfa7 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:52.149742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cf08d-f0e6-444c-a69d-29192ab8cc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:52.150555 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cf08d-f0e6-444c-a69d-29192ab8cc5d admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:52.150751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cf08d-f0e6-444c-a69d-29192ab8cc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:52.151029 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63cf08d-f0e6-444c-a69d-29192ab8cc5d admin admin] PATH_INFO: `/v3/projects/12350c50b0674c93861cb9250b9738a7/users/08365578ac624d3e8ae7a683b2c3b713/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:52.153023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cf08d-f0e6-444c-a69d-29192ab8cc5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=12350c50b0674c93861cb9250b9738a7, user_id=08365578ac624d3e8ae7a683b2c3b713, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:52.155817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63cf08d-f0e6-444c-a69d-29192ab8cc5d admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:52.167006 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1178/2396] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:02:52 2022] PUT /identity/v3/projects/12350c50b0674c93861cb9250b9738a7/users/08365578ac624d3e8ae7a683b2c3b713/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:02:52.182219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d11250-1771-414d-9b2e-5e8c37466794 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:52.182800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d11250-1771-414d-9b2e-5e8c37466794 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:52.183138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d11250-1771-414d-9b2e-5e8c37466794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:52.216165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-65d11250-1771-414d-9b2e-5e8c37466794 None None] MFA Rules not processed for user `08365578ac624d3e8ae7a683b2c3b713`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:02:52.217429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-65d11250-1771-414d-9b2e-5e8c37466794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:52.287738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1219/2397] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:02:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:02:52.299740 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cf08d-f0e6-444c-a69d-29192ab8cc5d admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:52.301456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d63cf08d-f0e6-444c-a69d-29192ab8cc5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:02:52.306976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:02:52.360537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63cf08d-f0e6-444c-a69d-29192ab8cc5d admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:52.364205 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff137d-fcd4-4da9-816e-c170aed7ad70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:52.368178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff137d-fcd4-4da9-816e-c170aed7ad70 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:52.368178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff137d-fcd4-4da9-816e-c170aed7ad70 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:52.368178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff137d-fcd4-4da9-816e-c170aed7ad70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:52.424083 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff137d-fcd4-4da9-816e-c170aed7ad70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:52.425696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff137d-fcd4-4da9-816e-c170aed7ad70 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:52.435094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1179/2398] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:02:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:02:54.485074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d11250-1771-414d-9b2e-5e8c37466794 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:54.494540 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d11250-1771-414d-9b2e-5e8c37466794 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:54.498103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17476fbd-143b-4b3d-8ba7-7951f47693f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:54.505997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17476fbd-143b-4b3d-8ba7-7951f47693f0 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:54.505997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17476fbd-143b-4b3d-8ba7-7951f47693f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:54.505997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17476fbd-143b-4b3d-8ba7-7951f47693f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:54.555669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17476fbd-143b-4b3d-8ba7-7951f47693f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:54.557090 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17476fbd-143b-4b3d-8ba7-7951f47693f0 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:54.566592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1220/2399] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:02:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:02:56.056491 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff137d-fcd4-4da9-816e-c170aed7ad70 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:02:56.064918 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff137d-fcd4-4da9-816e-c170aed7ad70 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:02:56.068514 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576faf51-6cff-404f-820c-508da79abed1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:02:56.069108 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576faf51-6cff-404f-820c-508da79abed1 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:02:56.069413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576faf51-6cff-404f-820c-508da79abed1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:02:56.069685 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576faf51-6cff-404f-820c-508da79abed1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:02:56.078689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576faf51-6cff-404f-820c-508da79abed1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:02:56.080097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576faf51-6cff-404f-820c-508da79abed1 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:02:56.089598 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1180/2400] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:02:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:00.755175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17476fbd-143b-4b3d-8ba7-7951f47693f0 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:00.817366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17476fbd-143b-4b3d-8ba7-7951f47693f0 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:00.821884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454a7d0-a3a0-4556-a9ed-926a403fb217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:00.822507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454a7d0-a3a0-4556-a9ed-926a403fb217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:00.822783 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454a7d0-a3a0-4556-a9ed-926a403fb217 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:00.823055 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454a7d0-a3a0-4556-a9ed-926a403fb217 admin admin] PATH_INFO: `/v3/users/edc06cf22707473eb7c0b3a343b892c7` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:00.824189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454a7d0-a3a0-4556-a9ed-926a403fb217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edc06cf22707473eb7c0b3a343b892c7)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:00.826216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454a7d0-a3a0-4556-a9ed-926a403fb217 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:00.912538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:01.028409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9454a7d0-a3a0-4556-a9ed-926a403fb217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edc06cf22707473eb7c0b3a343b892c7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:01.117131 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9454a7d0-a3a0-4556-a9ed-926a403fb217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edc06cf22707473eb7c0b3a343b892c7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:01.129871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9454a7d0-a3a0-4556-a9ed-926a403fb217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edc06cf22707473eb7c0b3a343b892c7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:01.130497 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9454a7d0-a3a0-4556-a9ed-926a403fb217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edc06cf22707473eb7c0b3a343b892c7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:01.131100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-9454a7d0-a3a0-4556-a9ed-926a403fb217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edc06cf22707473eb7c0b3a343b892c7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:01.143103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1221/2401] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:03:00 2022] DELETE /identity/v3/users/edc06cf22707473eb7c0b3a343b892c7 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:01.147727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df430ac4-5343-42d0-bf43-e53bcbad4df0 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:01.148337 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df430ac4-5343-42d0-bf43-e53bcbad4df0 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:01.148337 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df430ac4-5343-42d0-bf43-e53bcbad4df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:01.166987 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-df430ac4-5343-42d0-bf43-e53bcbad4df0 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:01.168173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-df430ac4-5343-42d0-bf43-e53bcbad4df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:01.228465 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1181/2402] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:01.238152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454a7d0-a3a0-4556-a9ed-926a403fb217 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:01.244785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9454a7d0-a3a0-4556-a9ed-926a403fb217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:01.296528 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454a7d0-a3a0-4556-a9ed-926a403fb217 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:01.300226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e0f83-05fe-467e-9425-fbd69504f695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:01.300829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e0f83-05fe-467e-9425-fbd69504f695 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:01.301105 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e0f83-05fe-467e-9425-fbd69504f695 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:01.301373 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e0f83-05fe-467e-9425-fbd69504f695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:01.311854 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e0f83-05fe-467e-9425-fbd69504f695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:01.313295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e0f83-05fe-467e-9425-fbd69504f695 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:01.323276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1222/2403] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:01.764761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df430ac4-5343-42d0-bf43-e53bcbad4df0 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:01.766622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-df430ac4-5343-42d0-bf43-e53bcbad4df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:01.779404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df430ac4-5343-42d0-bf43-e53bcbad4df0 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:01.786288 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:01.787068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:01.787348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:01.787618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] PATH_INFO: `/v3/projects/7cfdb98bde394ef390d0dea491dd8a1e` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:01.788769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cfdb98bde394ef390d0dea491dd8a1e)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:01.790833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:01.905418 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:01.912614 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cfdb98bde394ef390d0dea491dd8a1e', 'request_id': 'req-af2a77a6-92fe-410b-b255-b0705f28a9f0'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:01.918006 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cfdb98bde394ef390d0dea491dd8a1e', 'request_id': 'req-af2a77a6-92fe-410b-b255-b0705f28a9f0'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:01.925309 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cfdb98bde394ef390d0dea491dd8a1e', 'request_id': 'req-af2a77a6-92fe-410b-b255-b0705f28a9f0'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:02.042901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] The token cache is being invalidate because project 7cfdb98bde394ef390d0dea491dd8a1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:03:02.043191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:02.043848 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:02.045335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1182/2404] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:03:01 2022] DELETE /identity/v3/projects/7cfdb98bde394ef390d0dea491dd8a1e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:02.049928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e0f83-05fe-467e-9425-fbd69504f695 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:02.059619 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e0f83-05fe-467e-9425-fbd69504f695 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:02.115634 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb441f-ea4a-4290-8b54-398477882a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:02.116271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb441f-ea4a-4290-8b54-398477882a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:02.116583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb441f-ea4a-4290-8b54-398477882a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:02.116853 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb441f-ea4a-4290-8b54-398477882a2d admin admin] PATH_INFO: `/v3/users/2f88405b171f42e7a4b63c217129244f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:02.117983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb441f-ea4a-4290-8b54-398477882a2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f88405b171f42e7a4b63c217129244f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:02.119921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeb441f-ea4a-4290-8b54-398477882a2d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:02.366544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-beeb441f-ea4a-4290-8b54-398477882a2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f88405b171f42e7a4b63c217129244f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:02.368552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:02.448682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-beeb441f-ea4a-4290-8b54-398477882a2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f88405b171f42e7a4b63c217129244f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:02.461195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-beeb441f-ea4a-4290-8b54-398477882a2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f88405b171f42e7a4b63c217129244f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:02.461855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-beeb441f-ea4a-4290-8b54-398477882a2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f88405b171f42e7a4b63c217129244f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:02.462453 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-beeb441f-ea4a-4290-8b54-398477882a2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f88405b171f42e7a4b63c217129244f'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:02.474473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1223/2405] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:03:02 2022] DELETE /identity/v3/users/2f88405b171f42e7a4b63c217129244f => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:03.164937 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:03.170092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:03.179536 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a77a6-92fe-410b-b255-b0705f28a9f0 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:03.237740 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:03.238427 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:03.238715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:03.238988 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] PATH_INFO: `/v3/projects/e6e3f0fb2ffb48229c57f83e6018dac2` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:03.240149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6e3f0fb2ffb48229c57f83e6018dac2)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:03.242065 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:03.359042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6e3f0fb2ffb48229c57f83e6018dac2', 'request_id': 'req-d5793d96-4a1e-446e-aa91-94630d90029b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:03.364715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6e3f0fb2ffb48229c57f83e6018dac2', 'request_id': 'req-d5793d96-4a1e-446e-aa91-94630d90029b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:03.371046 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6e3f0fb2ffb48229c57f83e6018dac2', 'request_id': 'req-d5793d96-4a1e-446e-aa91-94630d90029b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:03.460836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] The token cache is being invalidate because project e6e3f0fb2ffb48229c57f83e6018dac2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:03:03.461183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:03.461826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:03.463287 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1183/2406] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:03:03 2022] DELETE /identity/v3/projects/e6e3f0fb2ffb48229c57f83e6018dac2 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:03.467473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb441f-ea4a-4290-8b54-398477882a2d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:03.469458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-beeb441f-ea4a-4290-8b54-398477882a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:03.483108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb441f-ea4a-4290-8b54-398477882a2d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:03.540060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d264f56-a5c1-4b74-989e-cdd55c9983ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:03.540683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d264f56-a5c1-4b74-989e-cdd55c9983ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:03.540956 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d264f56-a5c1-4b74-989e-cdd55c9983ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:03.541222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d264f56-a5c1-4b74-989e-cdd55c9983ef admin admin] PATH_INFO: `/v3/users/08365578ac624d3e8ae7a683b2c3b713` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:03.542483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d264f56-a5c1-4b74-989e-cdd55c9983ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=08365578ac624d3e8ae7a683b2c3b713)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:03.544402 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d264f56-a5c1-4b74-989e-cdd55c9983ef admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:03.731359 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2d264f56-a5c1-4b74-989e-cdd55c9983ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08365578ac624d3e8ae7a683b2c3b713'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:03.825971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2d264f56-a5c1-4b74-989e-cdd55c9983ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08365578ac624d3e8ae7a683b2c3b713'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:03.839050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2d264f56-a5c1-4b74-989e-cdd55c9983ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08365578ac624d3e8ae7a683b2c3b713'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:03.839700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2d264f56-a5c1-4b74-989e-cdd55c9983ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08365578ac624d3e8ae7a683b2c3b713'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:03.840314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-2d264f56-a5c1-4b74-989e-cdd55c9983ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08365578ac624d3e8ae7a683b2c3b713'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:03.852266 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1224/2407] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:03:03 2022] DELETE /identity/v3/users/08365578ac624d3e8ae7a683b2c3b713 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:04.272996 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:04.276313 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:04.287694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5793d96-4a1e-446e-aa91-94630d90029b admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:04.341348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:04.348106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825b973c-73b0-40d8-bc9d-075f694dc049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:04.348735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b973c-73b0-40d8-bc9d-075f694dc049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:04.349010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b973c-73b0-40d8-bc9d-075f694dc049 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:04.349278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825b973c-73b0-40d8-bc9d-075f694dc049 admin admin] PATH_INFO: `/v3/projects/12350c50b0674c93861cb9250b9738a7` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:04.350402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b973c-73b0-40d8-bc9d-075f694dc049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12350c50b0674c93861cb9250b9738a7)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:04.352330 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825b973c-73b0-40d8-bc9d-075f694dc049 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:04.473626 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-825b973c-73b0-40d8-bc9d-075f694dc049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12350c50b0674c93861cb9250b9738a7', 'request_id': 'req-825b973c-73b0-40d8-bc9d-075f694dc049'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:04.479489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-825b973c-73b0-40d8-bc9d-075f694dc049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12350c50b0674c93861cb9250b9738a7', 'request_id': 'req-825b973c-73b0-40d8-bc9d-075f694dc049'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:04.486467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-825b973c-73b0-40d8-bc9d-075f694dc049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12350c50b0674c93861cb9250b9738a7', 'request_id': 'req-825b973c-73b0-40d8-bc9d-075f694dc049'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:04.625882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-825b973c-73b0-40d8-bc9d-075f694dc049 admin admin] The token cache is being invalidate because project 12350c50b0674c93861cb9250b9738a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:03:04.626113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-825b973c-73b0-40d8-bc9d-075f694dc049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:04.626751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-825b973c-73b0-40d8-bc9d-075f694dc049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:04.628311 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1184/2408] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:03:04 2022] DELETE /identity/v3/projects/12350c50b0674c93861cb9250b9738a7 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:04.634353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b9ac8-cd80-4211-8b23-73b2a4caeae1 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:04.634858 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b9ac8-cd80-4211-8b23-73b2a4caeae1 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:04.634979 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b9ac8-cd80-4211-8b23-73b2a4caeae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:04.654656 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7d7b9ac8-cd80-4211-8b23-73b2a4caeae1 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:04.659447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7d7b9ac8-cd80-4211-8b23-73b2a4caeae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:04.695947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:04.715390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1225/2409] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:04.777396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561430b0-e099-4bf0-809e-1a3a5f1624b8 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:04.777678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561430b0-e099-4bf0-809e-1a3a5f1624b8 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:04.777991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561430b0-e099-4bf0-809e-1a3a5f1624b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:04.809760 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-561430b0-e099-4bf0-809e-1a3a5f1624b8 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:04.814039 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-561430b0-e099-4bf0-809e-1a3a5f1624b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:04.819797 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1185/2410] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:04.824236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b9ac8-cd80-4211-8b23-73b2a4caeae1 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:04.838012 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b9ac8-cd80-4211-8b23-73b2a4caeae1 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:04.843471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8dc9ab-a0e3-40b1-97ae-89e43702bea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:04.844454 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8dc9ab-a0e3-40b1-97ae-89e43702bea4 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:04.844454 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8dc9ab-a0e3-40b1-97ae-89e43702bea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:04.844769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8dc9ab-a0e3-40b1-97ae-89e43702bea4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:04.845417 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8dc9ab-a0e3-40b1-97ae-89e43702bea4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:03:04.845721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8dc9ab-a0e3-40b1-97ae-89e43702bea4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:04.847371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8dc9ab-a0e3-40b1-97ae-89e43702bea4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:04.859669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1226/2411] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:03:04.863810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561430b0-e099-4bf0-809e-1a3a5f1624b8 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:04.873623 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561430b0-e099-4bf0-809e-1a3a5f1624b8 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:04.879067 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3f9e2-e7c8-467d-8a7e-c9faa7ffd5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:04.879216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3f9e2-e7c8-467d-8a7e-c9faa7ffd5a0 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:04.879504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3f9e2-e7c8-467d-8a7e-c9faa7ffd5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:04.879780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3f9e2-e7c8-467d-8a7e-c9faa7ffd5a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:04.880426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3f9e2-e7c8-467d-8a7e-c9faa7ffd5a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:04.882665 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3f9e2-e7c8-467d-8a7e-c9faa7ffd5a0 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:05.011049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1186/2412] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:03:04 2022] POST /identity/v3/projects => generated 395 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:05.016164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8dc9ab-a0e3-40b1-97ae-89e43702bea4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:05.025960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8dc9ab-a0e3-40b1-97ae-89e43702bea4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:05.082770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a45ef5-181e-49c4-a8d9-1abfad1d1c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:05.083305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a45ef5-181e-49c4-a8d9-1abfad1d1c99 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:05.083586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a45ef5-181e-49c4-a8d9-1abfad1d1c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:05.083891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a45ef5-181e-49c4-a8d9-1abfad1d1c99 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:05.084511 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a45ef5-181e-49c4-a8d9-1abfad1d1c99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:05.086742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a45ef5-181e-49c4-a8d9-1abfad1d1c99 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:05.209685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1227/2413] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:03:05 2022] POST /identity/v3/users => generated 298 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:03:05.214288 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3f9e2-e7c8-467d-8a7e-c9faa7ffd5a0 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:05.224774 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3f9e2-e7c8-467d-8a7e-c9faa7ffd5a0 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:05.229369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9db5d-f137-43fa-864c-f673b19147a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:05.230007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9db5d-f137-43fa-864c-f673b19147a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:05.230289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9db5d-f137-43fa-864c-f673b19147a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:05.230593 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9db5d-f137-43fa-864c-f673b19147a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:05.231311 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9db5d-f137-43fa-864c-f673b19147a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:05.232905 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9db5d-f137-43fa-864c-f673b19147a8 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:05.243080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1187/2414] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:05.247106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a45ef5-181e-49c4-a8d9-1abfad1d1c99 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:05.257702 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a45ef5-181e-49c4-a8d9-1abfad1d1c99 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:05.262747 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d06998-9bd8-4603-933e-0e06e7c04369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:05.263346 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d06998-9bd8-4603-933e-0e06e7c04369 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:05.263623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d06998-9bd8-4603-933e-0e06e7c04369 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:05.263911 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d06998-9bd8-4603-933e-0e06e7c04369 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:05.265176 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d06998-9bd8-4603-933e-0e06e7c04369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:05.266834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d06998-9bd8-4603-933e-0e06e7c04369 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:05.277579 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1228/2415] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:05.281611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9db5d-f137-43fa-864c-f673b19147a8 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:05.291568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9db5d-f137-43fa-864c-f673b19147a8 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:05.296084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54c780-c03a-405b-9e2f-7e7fd6676427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:05.296724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54c780-c03a-405b-9e2f-7e7fd6676427 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:05.297001 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54c780-c03a-405b-9e2f-7e7fd6676427 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:05.297274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54c780-c03a-405b-9e2f-7e7fd6676427 admin admin] PATH_INFO: `/v3/projects/bd7b621fdf8d4438b473cf2c0d7597ee/users/f3fa5ed342aa4290a2171fc4ceb50714/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:05.313382 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54c780-c03a-405b-9e2f-7e7fd6676427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd7b621fdf8d4438b473cf2c0d7597ee, user_id=f3fa5ed342aa4290a2171fc4ceb50714, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:05.316867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54c780-c03a-405b-9e2f-7e7fd6676427 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:05.393188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1188/2416] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:05 2022] PUT /identity/v3/projects/bd7b621fdf8d4438b473cf2c0d7597ee/users/f3fa5ed342aa4290a2171fc4ceb50714/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:05.403431 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f586d-b08d-49cd-ab63-8d1ca741622a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:05.403713 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f586d-b08d-49cd-ab63-8d1ca741622a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:05.403984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f586d-b08d-49cd-ab63-8d1ca741622a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:05.432299 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a14f586d-b08d-49cd-ab63-8d1ca741622a None None] MFA Rules not processed for user `f3fa5ed342aa4290a2171fc4ceb50714`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:05.433541 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a14f586d-b08d-49cd-ab63-8d1ca741622a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:05.500079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1229/2417] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:05 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:05.511164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54c780-c03a-405b-9e2f-7e7fd6676427 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:05.512863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6f54c780-c03a-405b-9e2f-7e7fd6676427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:05.570188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54c780-c03a-405b-9e2f-7e7fd6676427 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:05.573848 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bee389-63bd-4874-9a60-49bcc4ec41be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:05.574443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bee389-63bd-4874-9a60-49bcc4ec41be service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:05.574716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bee389-63bd-4874-9a60-49bcc4ec41be service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:05.574982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bee389-63bd-4874-9a60-49bcc4ec41be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:05.583136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bee389-63bd-4874-9a60-49bcc4ec41be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:05.584553 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bee389-63bd-4874-9a60-49bcc4ec41be service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:05.594104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1189/2418] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:05 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:14.907384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f586d-b08d-49cd-ab63-8d1ca741622a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:14.950576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:14.970800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f586d-b08d-49cd-ab63-8d1ca741622a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:14.975580 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249abd4-a5c5-4624-a605-2c5df2be087f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:14.981341 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249abd4-a5c5-4624-a605-2c5df2be087f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:14.981341 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249abd4-a5c5-4624-a605-2c5df2be087f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:14.981341 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5249abd4-a5c5-4624-a605-2c5df2be087f admin admin] PATH_INFO: `/v3/users/f3fa5ed342aa4290a2171fc4ceb50714` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:14.981341 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5249abd4-a5c5-4624-a605-2c5df2be087f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3fa5ed342aa4290a2171fc4ceb50714)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:14.981341 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5249abd4-a5c5-4624-a605-2c5df2be087f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:15.266115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5249abd4-a5c5-4624-a605-2c5df2be087f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3fa5ed342aa4290a2171fc4ceb50714'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:15.350973 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5249abd4-a5c5-4624-a605-2c5df2be087f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3fa5ed342aa4290a2171fc4ceb50714'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:15.364083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5249abd4-a5c5-4624-a605-2c5df2be087f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3fa5ed342aa4290a2171fc4ceb50714'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:15.364774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5249abd4-a5c5-4624-a605-2c5df2be087f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3fa5ed342aa4290a2171fc4ceb50714'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:15.365521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-5249abd4-a5c5-4624-a605-2c5df2be087f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3fa5ed342aa4290a2171fc4ceb50714'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:15.378258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1230/2419] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:03:14 2022] DELETE /identity/v3/users/f3fa5ed342aa4290a2171fc4ceb50714 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:15.382618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dc5ad-0c47-4ac9-883b-886d648e81ac None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:15.383108 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dc5ad-0c47-4ac9-883b-886d648e81ac None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:15.383215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dc5ad-0c47-4ac9-883b-886d648e81ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:15.402102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-0e2dc5ad-0c47-4ac9-883b-886d648e81ac None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:15.403302 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0e2dc5ad-0c47-4ac9-883b-886d648e81ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:15.422567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:15.461346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1190/2420] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:15.471017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249abd4-a5c5-4624-a605-2c5df2be087f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:15.472908 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5249abd4-a5c5-4624-a605-2c5df2be087f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:15.528526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5249abd4-a5c5-4624-a605-2c5df2be087f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:15.532110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11b2d9-96b9-40a5-837b-fe44d4dd137a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:15.532706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11b2d9-96b9-40a5-837b-fe44d4dd137a service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:15.532982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11b2d9-96b9-40a5-837b-fe44d4dd137a service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:15.533250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11b2d9-96b9-40a5-837b-fe44d4dd137a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:15.542644 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11b2d9-96b9-40a5-837b-fe44d4dd137a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:15.544047 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db11b2d9-96b9-40a5-837b-fe44d4dd137a service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:15.553902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1231/2421] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:15.965136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dc5ad-0c47-4ac9-883b-886d648e81ac None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:15.970894 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0e2dc5ad-0c47-4ac9-883b-886d648e81ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:15.979818 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dc5ad-0c47-4ac9-883b-886d648e81ac None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:15.984563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:15.985315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:15.985600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:15.985907 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] PATH_INFO: `/v3/projects/bd7b621fdf8d4438b473cf2c0d7597ee` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:15.987035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd7b621fdf8d4438b473cf2c0d7597ee)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:15.988978 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:16.123674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd7b621fdf8d4438b473cf2c0d7597ee', 'request_id': 'req-175e7db4-be35-4242-9140-62f9f98c0cda'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:16.128992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd7b621fdf8d4438b473cf2c0d7597ee', 'request_id': 'req-175e7db4-be35-4242-9140-62f9f98c0cda'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:16.135210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd7b621fdf8d4438b473cf2c0d7597ee', 'request_id': 'req-175e7db4-be35-4242-9140-62f9f98c0cda'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:16.219605 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] The token cache is being invalidate because project bd7b621fdf8d4438b473cf2c0d7597ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:03:16.219894 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:16.220564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:16.222140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1191/2422] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:03:15 2022] DELETE /identity/v3/projects/bd7b621fdf8d4438b473cf2c0d7597ee => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:18.417617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11b2d9-96b9-40a5-837b-fe44d4dd137a service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:18.420865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-db11b2d9-96b9-40a5-837b-fe44d4dd137a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:18.430886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11b2d9-96b9-40a5-837b-fe44d4dd137a service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:18.480894 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c4b83-8bc8-4858-b644-521dd6ffeda8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:18.481485 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c4b83-8bc8-4858-b644-521dd6ffeda8 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:18.481759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c4b83-8bc8-4858-b644-521dd6ffeda8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:18.482088 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1c4b83-8bc8-4858-b644-521dd6ffeda8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:18.483732 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1c1c4b83-8bc8-4858-b644-521dd6ffeda8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:18.496623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:18.544575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c4b83-8bc8-4858-b644-521dd6ffeda8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:18.546029 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1c4b83-8bc8-4858-b644-521dd6ffeda8 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:18.555655 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1232/2423] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:21.881125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:21.882796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:21.896923 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175e7db4-be35-4242-9140-62f9f98c0cda admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:21.901622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a9f99b-a46b-4a4b-b4bc-c35920469a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:21.902270 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9f99b-a46b-4a4b-b4bc-c35920469a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:21.902544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9f99b-a46b-4a4b-b4bc-c35920469a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:21.902813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a9f99b-a46b-4a4b-b4bc-c35920469a1e admin admin] PATH_INFO: `/v3/users/268dbf54f29c40d9a4daa772c44735d4` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:21.903948 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9f99b-a46b-4a4b-b4bc-c35920469a1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=268dbf54f29c40d9a4daa772c44735d4)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:21.905929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a9f99b-a46b-4a4b-b4bc-c35920469a1e admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:22.119689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-11a9f99b-a46b-4a4b-b4bc-c35920469a1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '268dbf54f29c40d9a4daa772c44735d4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:22.132017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-11a9f99b-a46b-4a4b-b4bc-c35920469a1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '268dbf54f29c40d9a4daa772c44735d4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:22.226671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-11a9f99b-a46b-4a4b-b4bc-c35920469a1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '268dbf54f29c40d9a4daa772c44735d4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:22.227297 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-11a9f99b-a46b-4a4b-b4bc-c35920469a1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '268dbf54f29c40d9a4daa772c44735d4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:22.234646 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-11a9f99b-a46b-4a4b-b4bc-c35920469a1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '268dbf54f29c40d9a4daa772c44735d4'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:22.241287 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1192/2424] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:03:21 2022] DELETE /identity/v3/users/268dbf54f29c40d9a4daa772c44735d4 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:22.492227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c4b83-8bc8-4858-b644-521dd6ffeda8 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:22.501141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1c4b83-8bc8-4858-b644-521dd6ffeda8 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:22.560471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:22.561099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:22.561372 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:22.561639 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c admin admin] PATH_INFO: `/v3/projects/e52433815b1440de8bfe05dc7d14b7f5` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:22.562760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e52433815b1440de8bfe05dc7d14b7f5)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:22.564665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:22.695144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e52433815b1440de8bfe05dc7d14b7f5', 'request_id': 'req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:22.769831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e52433815b1440de8bfe05dc7d14b7f5', 'request_id': 'req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:22.776596 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e52433815b1440de8bfe05dc7d14b7f5', 'request_id': 'req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:22.787532 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c admin admin] The token cache is being invalidate because project e52433815b1440de8bfe05dc7d14b7f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:03:22.787818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:22.788450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f5fd6fb3-c0df-4a46-8846-7cfa56f3884c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:22.789948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1233/2425] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:03:22 2022] DELETE /identity/v3/projects/e52433815b1440de8bfe05dc7d14b7f5 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:22.794940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91568a-b836-4e9d-a03e-3ccc44ffe7e1 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:22.795228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91568a-b836-4e9d-a03e-3ccc44ffe7e1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:22.795502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b91568a-b836-4e9d-a03e-3ccc44ffe7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:22.814636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-2b91568a-b836-4e9d-a03e-3ccc44ffe7e1 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:22.815837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2b91568a-b836-4e9d-a03e-3ccc44ffe7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:22.832648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:22.874943 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1193/2426] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:22.882556 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eadba1-bf73-4aa9-89b0-9d3d7e17111a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:22.882835 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eadba1-bf73-4aa9-89b0-9d3d7e17111a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:22.883166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eadba1-bf73-4aa9-89b0-9d3d7e17111a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:22.903277 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-80eadba1-bf73-4aa9-89b0-9d3d7e17111a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:22.904493 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-80eadba1-bf73-4aa9-89b0-9d3d7e17111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:22.916689 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1234/2427] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:22.917577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91568a-b836-4e9d-a03e-3ccc44ffe7e1 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:22.928112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b91568a-b836-4e9d-a03e-3ccc44ffe7e1 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:22.932646 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d3aa5-1818-4c18-898b-412c85116713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:22.933244 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d3aa5-1818-4c18-898b-412c85116713 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:22.933544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d3aa5-1818-4c18-898b-412c85116713 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:22.933857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d3aa5-1818-4c18-898b-412c85116713 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:22.940414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d3aa5-1818-4c18-898b-412c85116713 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:03:22.940414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d3aa5-1818-4c18-898b-412c85116713 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:22.940414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d3aa5-1818-4c18-898b-412c85116713 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:22.948277 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1194/2428] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 20:03:22.952493 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eadba1-bf73-4aa9-89b0-9d3d7e17111a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:22.965235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eadba1-bf73-4aa9-89b0-9d3d7e17111a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:22.970043 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7bdd3-e59e-4421-9414-2d28253696ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:22.970643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7bdd3-e59e-4421-9414-2d28253696ad admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:22.970924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7bdd3-e59e-4421-9414-2d28253696ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:22.971194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f7bdd3-e59e-4421-9414-2d28253696ad admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:22.971866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7bdd3-e59e-4421-9414-2d28253696ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:22.973748 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f7bdd3-e59e-4421-9414-2d28253696ad admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:23.083107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1235/2429] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:03:22 2022] POST /identity/v3/projects => generated 403 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:23.084302 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d3aa5-1818-4c18-898b-412c85116713 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:23.095132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d3aa5-1818-4c18-898b-412c85116713 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:23.152159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20efb4a-b5e3-4d8c-90a7-3e1b33f8beab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:23.152790 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20efb4a-b5e3-4d8c-90a7-3e1b33f8beab admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:23.153064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20efb4a-b5e3-4d8c-90a7-3e1b33f8beab admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:23.153331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20efb4a-b5e3-4d8c-90a7-3e1b33f8beab admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:23.153982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20efb4a-b5e3-4d8c-90a7-3e1b33f8beab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:23.155865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20efb4a-b5e3-4d8c-90a7-3e1b33f8beab admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:23.249125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1195/2430] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:03:23 2022] POST /identity/v3/users => generated 302 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:23.253758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7bdd3-e59e-4421-9414-2d28253696ad admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:23.265007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f7bdd3-e59e-4421-9414-2d28253696ad admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:23.270245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695a6c3-04df-43db-a311-383b7b1a7190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:23.270828 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695a6c3-04df-43db-a311-383b7b1a7190 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:23.271105 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695a6c3-04df-43db-a311-383b7b1a7190 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:23.271375 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695a6c3-04df-43db-a311-383b7b1a7190 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:23.272003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f695a6c3-04df-43db-a311-383b7b1a7190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:23.273582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f695a6c3-04df-43db-a311-383b7b1a7190 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:23.283411 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1236/2431] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:03:23.287267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20efb4a-b5e3-4d8c-90a7-3e1b33f8beab admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:23.297048 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20efb4a-b5e3-4d8c-90a7-3e1b33f8beab admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:23.302390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea7afa-bd9a-4f45-a03b-06389d0b722a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:23.302981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea7afa-bd9a-4f45-a03b-06389d0b722a admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:23.303259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea7afa-bd9a-4f45-a03b-06389d0b722a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:23.303523 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea7afa-bd9a-4f45-a03b-06389d0b722a admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:23.304145 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea7afa-bd9a-4f45-a03b-06389d0b722a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:23.305760 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea7afa-bd9a-4f45-a03b-06389d0b722a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:23.316942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1196/2432] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:23.321191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695a6c3-04df-43db-a311-383b7b1a7190 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:23.326567 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:23.331825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695a6c3-04df-43db-a311-383b7b1a7190 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:23.337783 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22c99e-fd00-4313-9318-b2e5db936267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:23.338512 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22c99e-fd00-4313-9318-b2e5db936267 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:23.338923 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22c99e-fd00-4313-9318-b2e5db936267 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:23.339203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22c99e-fd00-4313-9318-b2e5db936267 admin admin] PATH_INFO: `/v3/projects/84cd82d422304ec69522a3f0b35b0dbd/users/3610ae07275e42f29387884c8876a905/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:23.354133 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22c99e-fd00-4313-9318-b2e5db936267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84cd82d422304ec69522a3f0b35b0dbd, user_id=3610ae07275e42f29387884c8876a905, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:23.356771 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d22c99e-fd00-4313-9318-b2e5db936267 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:23.432661 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1237/2433] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:23 2022] PUT /identity/v3/projects/84cd82d422304ec69522a3f0b35b0dbd/users/3610ae07275e42f29387884c8876a905/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:03:23.444555 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea31185-d2b9-4027-a099-fa11a4cd9208 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:23.444839 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea31185-d2b9-4027-a099-fa11a4cd9208 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:23.445109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea31185-d2b9-4027-a099-fa11a4cd9208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:23.472636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-dea31185-d2b9-4027-a099-fa11a4cd9208 None None] MFA Rules not processed for user `3610ae07275e42f29387884c8876a905`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:23.473878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-dea31185-d2b9-4027-a099-fa11a4cd9208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:23.539906 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1197/2434] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:23 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:23.544346 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22c99e-fd00-4313-9318-b2e5db936267 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:23.606708 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22c99e-fd00-4313-9318-b2e5db936267 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:23.612249 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06093627-2aee-4ac0-88ff-2f7d8f45ff58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:23.612856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06093627-2aee-4ac0-88ff-2f7d8f45ff58 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:23.613133 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06093627-2aee-4ac0-88ff-2f7d8f45ff58 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:23.613407 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06093627-2aee-4ac0-88ff-2f7d8f45ff58 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:23.614129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06093627-2aee-4ac0-88ff-2f7d8f45ff58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:23.616025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06093627-2aee-4ac0-88ff-2f7d8f45ff58 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:23.714609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1238/2435] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:03:23 2022] POST /identity/v3/projects => generated 403 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:23.718523 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea31185-d2b9-4027-a099-fa11a4cd9208 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:23.729382 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea31185-d2b9-4027-a099-fa11a4cd9208 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:23.741673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:23.785032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a3694-5a84-48fc-a604-f37af5a151a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:23.788119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a3694-5a84-48fc-a604-f37af5a151a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:23.788119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a3694-5a84-48fc-a604-f37af5a151a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:23.788119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a3694-5a84-48fc-a604-f37af5a151a1 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:23.788119 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a3694-5a84-48fc-a604-f37af5a151a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:23.788907 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a3694-5a84-48fc-a604-f37af5a151a1 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:23.883963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1198/2436] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:03:23 2022] POST /identity/v3/users => generated 308 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:23.897224 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06093627-2aee-4ac0-88ff-2f7d8f45ff58 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:23.899640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06093627-2aee-4ac0-88ff-2f7d8f45ff58 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:23.904662 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b22e99-666b-4823-81b0-ba41076f11f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:23.905237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b22e99-666b-4823-81b0-ba41076f11f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:23.905517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b22e99-666b-4823-81b0-ba41076f11f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:23.905790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b22e99-666b-4823-81b0-ba41076f11f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:23.906452 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b22e99-666b-4823-81b0-ba41076f11f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:23.908054 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b22e99-666b-4823-81b0-ba41076f11f8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:23.918460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1239/2437] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:23.922695 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a3694-5a84-48fc-a604-f37af5a151a1 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:23.932701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a3694-5a84-48fc-a604-f37af5a151a1 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:23.937334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a90c3e-ab77-425a-947c-792892e85ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:23.937928 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a90c3e-ab77-425a-947c-792892e85ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:23.938208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a90c3e-ab77-425a-947c-792892e85ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:23.938483 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a90c3e-ab77-425a-947c-792892e85ffc admin admin] PATH_INFO: `/v3/projects/84a65c8611164139894c7a4d94374f06/users/e0b4352d8c9a4227b5c6043ce9c654f2/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:23.953325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a90c3e-ab77-425a-947c-792892e85ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a65c8611164139894c7a4d94374f06, user_id=e0b4352d8c9a4227b5c6043ce9c654f2, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:23.956666 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a90c3e-ab77-425a-947c-792892e85ffc admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:24.031477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1199/2438] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:23 2022] PUT /identity/v3/projects/84a65c8611164139894c7a4d94374f06/users/e0b4352d8c9a4227b5c6043ce9c654f2/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:24.035416 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b22e99-666b-4823-81b0-ba41076f11f8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:24.045562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b22e99-666b-4823-81b0-ba41076f11f8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:24.100707 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e59b08-9c95-4b08-a936-3a000af1b7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:24.101331 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e59b08-9c95-4b08-a936-3a000af1b7db admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:24.101605 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e59b08-9c95-4b08-a936-3a000af1b7db admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:24.101910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e59b08-9c95-4b08-a936-3a000af1b7db admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:24.102539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e59b08-9c95-4b08-a936-3a000af1b7db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:24.104098 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e59b08-9c95-4b08-a936-3a000af1b7db admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:24.114040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1240/2439] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:24.118106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a90c3e-ab77-425a-947c-792892e85ffc admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:24.128678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a90c3e-ab77-425a-947c-792892e85ffc admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:24.133093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e13b01-2ef4-4742-b984-d4a52f2d507e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:24.133724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e13b01-2ef4-4742-b984-d4a52f2d507e admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:24.134032 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e13b01-2ef4-4742-b984-d4a52f2d507e admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:24.134303 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e13b01-2ef4-4742-b984-d4a52f2d507e admin admin] PATH_INFO: `/v3/projects/84a65c8611164139894c7a4d94374f06/users/e0b4352d8c9a4227b5c6043ce9c654f2/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:24.136157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e13b01-2ef4-4742-b984-d4a52f2d507e admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a65c8611164139894c7a4d94374f06, user_id=e0b4352d8c9a4227b5c6043ce9c654f2, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:24.138814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e13b01-2ef4-4742-b984-d4a52f2d507e admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:24.149003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1200/2440] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:24 2022] PUT /identity/v3/projects/84a65c8611164139894c7a4d94374f06/users/e0b4352d8c9a4227b5c6043ce9c654f2/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:03:24.159144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490f187-44e0-49d4-96ff-c78ac4716733 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:24.159479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490f187-44e0-49d4-96ff-c78ac4716733 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:24.159751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0490f187-44e0-49d4-96ff-c78ac4716733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:24.188595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-0490f187-44e0-49d4-96ff-c78ac4716733 None None] MFA Rules not processed for user `e0b4352d8c9a4227b5c6043ce9c654f2`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:24.190531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0490f187-44e0-49d4-96ff-c78ac4716733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:24.227535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:24.257087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1241/2441] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:24 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:24.267817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e13b01-2ef4-4742-b984-d4a52f2d507e admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:24.273401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e0e13b01-2ef4-4742-b984-d4a52f2d507e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:24.326761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e13b01-2ef4-4742-b984-d4a52f2d507e admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:24.330474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e3206-1b6c-48e3-ad7d-b54776186db3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:24.331073 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e3206-1b6c-48e3-ad7d-b54776186db3 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:24.331349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e3206-1b6c-48e3-ad7d-b54776186db3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:24.331620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e3206-1b6c-48e3-ad7d-b54776186db3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:24.340727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e3206-1b6c-48e3-ad7d-b54776186db3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:24.342186 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e3206-1b6c-48e3-ad7d-b54776186db3 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:24.351420 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1201/2442] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:24 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:24.494517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490f187-44e0-49d4-96ff-c78ac4716733 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:24.555886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0490f187-44e0-49d4-96ff-c78ac4716733 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:24.560509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8c0a4-c980-4216-bd69-15497f794867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:24.561137 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8c0a4-c980-4216-bd69-15497f794867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:24.561416 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8c0a4-c980-4216-bd69-15497f794867 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:24.561688 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8c0a4-c980-4216-bd69-15497f794867 admin admin] PATH_INFO: `/v3/users/3610ae07275e42f29387884c8876a905` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:24.562855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8c0a4-c980-4216-bd69-15497f794867 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3610ae07275e42f29387884c8876a905)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:24.564768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8c0a4-c980-4216-bd69-15497f794867 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:24.737576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-82e8c0a4-c980-4216-bd69-15497f794867 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3610ae07275e42f29387884c8876a905'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:24.809289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-82e8c0a4-c980-4216-bd69-15497f794867 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3610ae07275e42f29387884c8876a905'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:24.822347 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-82e8c0a4-c980-4216-bd69-15497f794867 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3610ae07275e42f29387884c8876a905'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:24.823060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-82e8c0a4-c980-4216-bd69-15497f794867 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3610ae07275e42f29387884c8876a905'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:24.823668 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-82e8c0a4-c980-4216-bd69-15497f794867 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3610ae07275e42f29387884c8876a905'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:24.841089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1242/2443] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:03:24 2022] DELETE /identity/v3/users/3610ae07275e42f29387884c8876a905 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:24.845549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bc3b4-f3d0-4449-b6d1-ec78e4f66e86 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:24.845853 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bc3b4-f3d0-4449-b6d1-ec78e4f66e86 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:24.846763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2bc3b4-f3d0-4449-b6d1-ec78e4f66e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:24.867339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-7e2bc3b4-f3d0-4449-b6d1-ec78e4f66e86 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:24.868538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-7e2bc3b4-f3d0-4449-b6d1-ec78e4f66e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:24.926940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1202/2444] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:24.936453 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8c0a4-c980-4216-bd69-15497f794867 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:24.938023 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-82e8c0a4-c980-4216-bd69-15497f794867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:24.996232 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8c0a4-c980-4216-bd69-15497f794867 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:24.999907 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c667b00-7747-4280-ac98-99010ae7b9d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:25.000511 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c667b00-7747-4280-ac98-99010ae7b9d6 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:25.000787 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c667b00-7747-4280-ac98-99010ae7b9d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:25.001059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c667b00-7747-4280-ac98-99010ae7b9d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:25.008776 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:25.011159 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c667b00-7747-4280-ac98-99010ae7b9d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:25.012603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c667b00-7747-4280-ac98-99010ae7b9d6 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:25.022415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1243/2445] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:25.857378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bc3b4-f3d0-4449-b6d1-ec78e4f66e86 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:25.859603 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-7e2bc3b4-f3d0-4449-b6d1-ec78e4f66e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:25.894616 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2bc3b4-f3d0-4449-b6d1-ec78e4f66e86 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:25.894616 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:25.894616 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:25.894616 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:25.895161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] PATH_INFO: `/v3/projects/84cd82d422304ec69522a3f0b35b0dbd` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:25.895161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] RBAC: Authorizing `identity:delete_project(project_id=84cd82d422304ec69522a3f0b35b0dbd)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:25.895161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:25.901709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:25.993921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84cd82d422304ec69522a3f0b35b0dbd', 'request_id': 'req-83b49247-1877-4147-8f7d-98d8bfc7a6df'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:25.999651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84cd82d422304ec69522a3f0b35b0dbd', 'request_id': 'req-83b49247-1877-4147-8f7d-98d8bfc7a6df'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:26.007665 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84cd82d422304ec69522a3f0b35b0dbd', 'request_id': 'req-83b49247-1877-4147-8f7d-98d8bfc7a6df'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:26.117538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] The token cache is being invalidate because project 84cd82d422304ec69522a3f0b35b0dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:03:26.117917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:26.118555 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:26.120184 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1203/2446] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:03:25 2022] DELETE /identity/v3/projects/84cd82d422304ec69522a3f0b35b0dbd => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:26.124183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c667b00-7747-4280-ac98-99010ae7b9d6 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:26.134832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c667b00-7747-4280-ac98-99010ae7b9d6 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:26.204575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc2a33-849b-45d9-ad58-e425248c3a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:26.206985 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc2a33-849b-45d9-ad58-e425248c3a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:26.206985 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc2a33-849b-45d9-ad58-e425248c3a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:26.206985 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc2a33-849b-45d9-ad58-e425248c3a57 admin admin] PATH_INFO: `/v3/users/e0b4352d8c9a4227b5c6043ce9c654f2` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:26.208309 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc2a33-849b-45d9-ad58-e425248c3a57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b4352d8c9a4227b5c6043ce9c654f2)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:26.210927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc2a33-849b-45d9-ad58-e425248c3a57 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:26.492829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dffc2a33-849b-45d9-ad58-e425248c3a57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b4352d8c9a4227b5c6043ce9c654f2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:26.569442 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dffc2a33-849b-45d9-ad58-e425248c3a57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b4352d8c9a4227b5c6043ce9c654f2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:26.584502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dffc2a33-849b-45d9-ad58-e425248c3a57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b4352d8c9a4227b5c6043ce9c654f2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:26.585353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dffc2a33-849b-45d9-ad58-e425248c3a57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b4352d8c9a4227b5c6043ce9c654f2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:26.586423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dffc2a33-849b-45d9-ad58-e425248c3a57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b4352d8c9a4227b5c6043ce9c654f2'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:26.600263 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1244/2447] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:03:26 2022] DELETE /identity/v3/users/e0b4352d8c9a4227b5c6043ce9c654f2 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:26.982338 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:26.984064 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:26.999404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b49247-1877-4147-8f7d-98d8bfc7a6df admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:27.084153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e3ceb-6322-4bb9-be3b-43c0296e7261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:27.084153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e3ceb-6322-4bb9-be3b-43c0296e7261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:27.084153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e3ceb-6322-4bb9-be3b-43c0296e7261 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:27.084153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e3ceb-6322-4bb9-be3b-43c0296e7261 admin admin] PATH_INFO: `/v3/projects/84a65c8611164139894c7a4d94374f06` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:27.084527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e3ceb-6322-4bb9-be3b-43c0296e7261 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84a65c8611164139894c7a4d94374f06)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:27.084527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e3ceb-6322-4bb9-be3b-43c0296e7261 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:27.210045 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d73e3ceb-6322-4bb9-be3b-43c0296e7261 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a65c8611164139894c7a4d94374f06', 'request_id': 'req-d73e3ceb-6322-4bb9-be3b-43c0296e7261'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:27.215970 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d73e3ceb-6322-4bb9-be3b-43c0296e7261 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a65c8611164139894c7a4d94374f06', 'request_id': 'req-d73e3ceb-6322-4bb9-be3b-43c0296e7261'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:27.222595 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d73e3ceb-6322-4bb9-be3b-43c0296e7261 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a65c8611164139894c7a4d94374f06', 'request_id': 'req-d73e3ceb-6322-4bb9-be3b-43c0296e7261'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:27.329181 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d73e3ceb-6322-4bb9-be3b-43c0296e7261 admin admin] The token cache is being invalidate because project 84a65c8611164139894c7a4d94374f06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:03:27.329477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d73e3ceb-6322-4bb9-be3b-43c0296e7261 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:27.330131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d73e3ceb-6322-4bb9-be3b-43c0296e7261 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:27.331922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1204/2448] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:03:26 2022] DELETE /identity/v3/projects/84a65c8611164139894c7a4d94374f06 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:27.337902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559a93-ecb1-4803-b0e0-2eb3199e7a5c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:27.338437 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559a93-ecb1-4803-b0e0-2eb3199e7a5c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:27.339313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559a93-ecb1-4803-b0e0-2eb3199e7a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:27.363863 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-da559a93-ecb1-4803-b0e0-2eb3199e7a5c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:27.365182 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-da559a93-ecb1-4803-b0e0-2eb3199e7a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:27.426456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1245/2449] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:27.435332 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43241cf-af09-4119-a6ce-575178fe8a35 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:27.435618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43241cf-af09-4119-a6ce-575178fe8a35 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:27.436006 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43241cf-af09-4119-a6ce-575178fe8a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:27.457071 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-a43241cf-af09-4119-a6ce-575178fe8a35 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:27.458325 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a43241cf-af09-4119-a6ce-575178fe8a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:27.466635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1205/2450] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:27.470790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da559a93-ecb1-4803-b0e0-2eb3199e7a5c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:27.481106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da559a93-ecb1-4803-b0e0-2eb3199e7a5c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:27.485901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0514992-6767-4fa8-b743-6a770426f1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:27.486493 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0514992-6767-4fa8-b743-6a770426f1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:27.487469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0514992-6767-4fa8-b743-6a770426f1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:27.487744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0514992-6767-4fa8-b743-6a770426f1b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:27.488380 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0514992-6767-4fa8-b743-6a770426f1b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:03:27.488684 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0514992-6767-4fa8-b743-6a770426f1b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:27.490316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0514992-6767-4fa8-b743-6a770426f1b4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:27.501778 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1246/2451] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:03:27.506011 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43241cf-af09-4119-a6ce-575178fe8a35 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:27.515112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43241cf-af09-4119-a6ce-575178fe8a35 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:27.519810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8c9d9-80d5-47b0-883d-5adc21659874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:27.520392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8c9d9-80d5-47b0-883d-5adc21659874 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:27.520664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8c9d9-80d5-47b0-883d-5adc21659874 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:27.520930 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8c9d9-80d5-47b0-883d-5adc21659874 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:27.521539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8c9d9-80d5-47b0-883d-5adc21659874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:27.523469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8c9d9-80d5-47b0-883d-5adc21659874 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:27.678125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:27.685114 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1206/2452] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:03:27 2022] POST /identity/v3/projects => generated 357 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:27.689514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0514992-6767-4fa8-b743-6a770426f1b4 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:27.694703 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:27.699951 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0514992-6767-4fa8-b743-6a770426f1b4 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:27.755998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df7553-472a-4e8a-85b2-b7d84a0e6505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:27.756555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df7553-472a-4e8a-85b2-b7d84a0e6505 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:27.756834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df7553-472a-4e8a-85b2-b7d84a0e6505 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:27.757108 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11df7553-472a-4e8a-85b2-b7d84a0e6505 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:27.757733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df7553-472a-4e8a-85b2-b7d84a0e6505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:27.759679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11df7553-472a-4e8a-85b2-b7d84a0e6505 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:27.889271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1247/2453] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:03:27 2022] POST /identity/v3/users => generated 279 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:27.894168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8c9d9-80d5-47b0-883d-5adc21659874 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:27.904103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8c9d9-80d5-47b0-883d-5adc21659874 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:27.909181 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29648d-29f0-4340-95ce-88da07ee3c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:27.909772 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29648d-29f0-4340-95ce-88da07ee3c7b admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:27.910075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29648d-29f0-4340-95ce-88da07ee3c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:27.910342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29648d-29f0-4340-95ce-88da07ee3c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:27.910984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29648d-29f0-4340-95ce-88da07ee3c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:27.912697 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29648d-29f0-4340-95ce-88da07ee3c7b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:27.922999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1207/2454] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:27.927001 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df7553-472a-4e8a-85b2-b7d84a0e6505 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:27.936909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11df7553-472a-4e8a-85b2-b7d84a0e6505 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:27.941596 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a7a4e7-de7e-4af0-9dbe-e44e4726b162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:27.942223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7a4e7-de7e-4af0-9dbe-e44e4726b162 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:27.942524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7a4e7-de7e-4af0-9dbe-e44e4726b162 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:27.942783 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a7a4e7-de7e-4af0-9dbe-e44e4726b162 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:27.943414 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7a4e7-de7e-4af0-9dbe-e44e4726b162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:27.944993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a7a4e7-de7e-4af0-9dbe-e44e4726b162 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:27.955915 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1248/2455] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:27.959887 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29648d-29f0-4340-95ce-88da07ee3c7b admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:27.969911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29648d-29f0-4340-95ce-88da07ee3c7b admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:27.974922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd57fbf-45cb-4cfc-9a24-d5e611ba5d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:27.975554 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd57fbf-45cb-4cfc-9a24-d5e611ba5d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:27.975826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd57fbf-45cb-4cfc-9a24-d5e611ba5d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:27.976091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd57fbf-45cb-4cfc-9a24-d5e611ba5d6b admin admin] PATH_INFO: `/v3/projects/738ff63fc1774138abbbe2c91594de0a/users/059ab2513b834e16acae42b1fcf5b250/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:27.990286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd57fbf-45cb-4cfc-9a24-d5e611ba5d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=738ff63fc1774138abbbe2c91594de0a, user_id=059ab2513b834e16acae42b1fcf5b250, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:27.993023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd57fbf-45cb-4cfc-9a24-d5e611ba5d6b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:28.094476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1208/2456] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:27 2022] PUT /identity/v3/projects/738ff63fc1774138abbbe2c91594de0a/users/059ab2513b834e16acae42b1fcf5b250/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:28.105326 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b29f9-a12b-48cc-8d54-8e9d8b0a5f24 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:28.105613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b29f9-a12b-48cc-8d54-8e9d8b0a5f24 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:28.105917 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497b29f9-a12b-48cc-8d54-8e9d8b0a5f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:28.139863 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-497b29f9-a12b-48cc-8d54-8e9d8b0a5f24 None None] MFA Rules not processed for user `059ab2513b834e16acae42b1fcf5b250`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:28.139863 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-497b29f9-a12b-48cc-8d54-8e9d8b0a5f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:28.206414 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1249/2457] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:28 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:28.207583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd57fbf-45cb-4cfc-9a24-d5e611ba5d6b admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:28.270498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd57fbf-45cb-4cfc-9a24-d5e611ba5d6b admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:28.275650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49037e00-dfa9-4f6e-af1b-6f2680b7f99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:28.275931 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49037e00-dfa9-4f6e-af1b-6f2680b7f99a admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:28.277857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49037e00-dfa9-4f6e-af1b-6f2680b7f99a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:28.277857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49037e00-dfa9-4f6e-af1b-6f2680b7f99a admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:28.277857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49037e00-dfa9-4f6e-af1b-6f2680b7f99a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:28.278968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49037e00-dfa9-4f6e-af1b-6f2680b7f99a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:28.376159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1209/2458] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:03:28 2022] POST /identity/v3/projects => generated 355 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:03:28.380121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b29f9-a12b-48cc-8d54-8e9d8b0a5f24 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:28.390586 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497b29f9-a12b-48cc-8d54-8e9d8b0a5f24 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:28.445163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf994c3-aa26-404c-ab66-7b37c5cee06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:28.445774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf994c3-aa26-404c-ab66-7b37c5cee06e admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:28.446075 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf994c3-aa26-404c-ab66-7b37c5cee06e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:28.446423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf994c3-aa26-404c-ab66-7b37c5cee06e admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:28.446956 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf994c3-aa26-404c-ab66-7b37c5cee06e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:28.449142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf994c3-aa26-404c-ab66-7b37c5cee06e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:28.553214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:28.558478 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1250/2459] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:03:28 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:03:28.562505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49037e00-dfa9-4f6e-af1b-6f2680b7f99a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:28.573049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49037e00-dfa9-4f6e-af1b-6f2680b7f99a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:28.577719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc6e00-ba2f-4ed9-9e8d-7b71b3c90e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:28.578355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc6e00-ba2f-4ed9-9e8d-7b71b3c90e52 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:28.578628 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc6e00-ba2f-4ed9-9e8d-7b71b3c90e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:28.578898 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc6e00-ba2f-4ed9-9e8d-7b71b3c90e52 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:28.579525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc6e00-ba2f-4ed9-9e8d-7b71b3c90e52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:28.581608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edc6e00-ba2f-4ed9-9e8d-7b71b3c90e52 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:28.592420 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1210/2460] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:28.597278 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf994c3-aa26-404c-ab66-7b37c5cee06e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:28.607715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf994c3-aa26-404c-ab66-7b37c5cee06e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:28.614078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb6e1b-21f7-49b1-9910-4fb85d8d1ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:28.614078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb6e1b-21f7-49b1-9910-4fb85d8d1ce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:28.614078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb6e1b-21f7-49b1-9910-4fb85d8d1ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:28.614078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb6e1b-21f7-49b1-9910-4fb85d8d1ce5 admin admin] PATH_INFO: `/v3/projects/9436b434ca8c437eb3a87b79dc1718f1/users/63885726ab5044e99e755b0cbb3e05ac/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:28.629093 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb6e1b-21f7-49b1-9910-4fb85d8d1ce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9436b434ca8c437eb3a87b79dc1718f1, user_id=63885726ab5044e99e755b0cbb3e05ac, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:28.631719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb6e1b-21f7-49b1-9910-4fb85d8d1ce5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:28.729663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1251/2461] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:28 2022] PUT /identity/v3/projects/9436b434ca8c437eb3a87b79dc1718f1/users/63885726ab5044e99e755b0cbb3e05ac/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:03:28.733613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc6e00-ba2f-4ed9-9e8d-7b71b3c90e52 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:28.744278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc6e00-ba2f-4ed9-9e8d-7b71b3c90e52 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:28.800021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ce65e7-0fce-46f6-83eb-661e66ca5835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:28.800624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce65e7-0fce-46f6-83eb-661e66ca5835 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:28.800900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce65e7-0fce-46f6-83eb-661e66ca5835 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:28.801175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ce65e7-0fce-46f6-83eb-661e66ca5835 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:28.801860 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ce65e7-0fce-46f6-83eb-661e66ca5835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:28.803426 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ce65e7-0fce-46f6-83eb-661e66ca5835 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:28.813133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1211/2462] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:28.817471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb6e1b-21f7-49b1-9910-4fb85d8d1ce5 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:28.827507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb6e1b-21f7-49b1-9910-4fb85d8d1ce5 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:28.832570 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247e368-218c-4e1a-9c1e-3c35d4788324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:28.833206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247e368-218c-4e1a-9c1e-3c35d4788324 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:28.833482 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247e368-218c-4e1a-9c1e-3c35d4788324 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:28.833752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247e368-218c-4e1a-9c1e-3c35d4788324 admin admin] PATH_INFO: `/v3/projects/9436b434ca8c437eb3a87b79dc1718f1/users/63885726ab5044e99e755b0cbb3e05ac/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:28.835923 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247e368-218c-4e1a-9c1e-3c35d4788324 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9436b434ca8c437eb3a87b79dc1718f1, user_id=63885726ab5044e99e755b0cbb3e05ac, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:28.842589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247e368-218c-4e1a-9c1e-3c35d4788324 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:28.849090 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1252/2463] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:28 2022] PUT /identity/v3/projects/9436b434ca8c437eb3a87b79dc1718f1/users/63885726ab5044e99e755b0cbb3e05ac/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:28.859734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b5b24-6ad2-44fc-93f0-6055842e4171 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:28.860267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b5b24-6ad2-44fc-93f0-6055842e4171 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:28.860665 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11b5b24-6ad2-44fc-93f0-6055842e4171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:28.890662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e11b5b24-6ad2-44fc-93f0-6055842e4171 None None] MFA Rules not processed for user `63885726ab5044e99e755b0cbb3e05ac`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:28.891959 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e11b5b24-6ad2-44fc-93f0-6055842e4171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:28.919011 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:28.969667 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1212/2464] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:28.981285 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247e368-218c-4e1a-9c1e-3c35d4788324 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:28.983185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9247e368-218c-4e1a-9c1e-3c35d4788324 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:29.042225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247e368-218c-4e1a-9c1e-3c35d4788324 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:29.045910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29995981-99c8-4eae-b719-2d754eb1db34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:29.046514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29995981-99c8-4eae-b719-2d754eb1db34 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:29.046790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29995981-99c8-4eae-b719-2d754eb1db34 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:29.047061 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29995981-99c8-4eae-b719-2d754eb1db34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:29.055895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29995981-99c8-4eae-b719-2d754eb1db34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:29.057302 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29995981-99c8-4eae-b719-2d754eb1db34 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:29.067145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1253/2465] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:29.297326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b5b24-6ad2-44fc-93f0-6055842e4171 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:29.298734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e11b5b24-6ad2-44fc-93f0-6055842e4171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:29.311927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11b5b24-6ad2-44fc-93f0-6055842e4171 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:29.316154 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed0113-b19c-4980-93af-ad285316f535 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:29.316746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed0113-b19c-4980-93af-ad285316f535 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:29.317022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed0113-b19c-4980-93af-ad285316f535 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:29.317293 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed0113-b19c-4980-93af-ad285316f535 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:29.371445 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed0113-b19c-4980-93af-ad285316f535 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:29.372922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed0113-b19c-4980-93af-ad285316f535 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:29.382312 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1213/2466] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:29 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:31.667099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29995981-99c8-4eae-b719-2d754eb1db34 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:31.668657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-29995981-99c8-4eae-b719-2d754eb1db34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:31.726042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29995981-99c8-4eae-b719-2d754eb1db34 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:31.729648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d90db2-896d-4bdb-a1ef-329ce52af730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:31.730276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d90db2-896d-4bdb-a1ef-329ce52af730 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:31.730552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d90db2-896d-4bdb-a1ef-329ce52af730 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:31.730824 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d90db2-896d-4bdb-a1ef-329ce52af730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:31.732513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-32d90db2-896d-4bdb-a1ef-329ce52af730 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:31.793787 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d90db2-896d-4bdb-a1ef-329ce52af730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:31.795224 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d90db2-896d-4bdb-a1ef-329ce52af730 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:31.804726 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1254/2467] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:35.506419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed0113-b19c-4980-93af-ad285316f535 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:35.507977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e7ed0113-b19c-4980-93af-ad285316f535 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:35.523046 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed0113-b19c-4980-93af-ad285316f535 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:35.527901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b0b5f-9513-456e-89c0-b2ea590029b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:35.528530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b0b5f-9513-456e-89c0-b2ea590029b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:35.528805 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b0b5f-9513-456e-89c0-b2ea590029b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:35.529075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b0b5f-9513-456e-89c0-b2ea590029b6 admin admin] PATH_INFO: `/v3/users/2bf1c7a6cc2e4b1294e7b162a8aba738` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:35.530240 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b0b5f-9513-456e-89c0-b2ea590029b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bf1c7a6cc2e4b1294e7b162a8aba738)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:35.532162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b0b5f-9513-456e-89c0-b2ea590029b6 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:35.708938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4b8b0b5f-9513-456e-89c0-b2ea590029b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bf1c7a6cc2e4b1294e7b162a8aba738'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:35.722754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4b8b0b5f-9513-456e-89c0-b2ea590029b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bf1c7a6cc2e4b1294e7b162a8aba738'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:35.780627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4b8b0b5f-9513-456e-89c0-b2ea590029b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bf1c7a6cc2e4b1294e7b162a8aba738'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:35.781305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4b8b0b5f-9513-456e-89c0-b2ea590029b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bf1c7a6cc2e4b1294e7b162a8aba738'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:35.788741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4b8b0b5f-9513-456e-89c0-b2ea590029b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bf1c7a6cc2e4b1294e7b162a8aba738'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:35.794743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1214/2468] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:03:35 2022] DELETE /identity/v3/users/2bf1c7a6cc2e4b1294e7b162a8aba738 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:36.091985 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d90db2-896d-4bdb-a1ef-329ce52af730 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:36.103954 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d90db2-896d-4bdb-a1ef-329ce52af730 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:36.127653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:36.174661 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd640b0e-7842-4bd1-b08d-f63e423f9736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:36.175297 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd640b0e-7842-4bd1-b08d-f63e423f9736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:36.175589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd640b0e-7842-4bd1-b08d-f63e423f9736 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:36.175840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd640b0e-7842-4bd1-b08d-f63e423f9736 admin admin] PATH_INFO: `/v3/projects/bdb5ca35665545119e4ca1f379c37a73` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:36.176943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd640b0e-7842-4bd1-b08d-f63e423f9736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdb5ca35665545119e4ca1f379c37a73)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:36.179415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd640b0e-7842-4bd1-b08d-f63e423f9736 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:36.289790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fd640b0e-7842-4bd1-b08d-f63e423f9736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdb5ca35665545119e4ca1f379c37a73', 'request_id': 'req-fd640b0e-7842-4bd1-b08d-f63e423f9736'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:36.368674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fd640b0e-7842-4bd1-b08d-f63e423f9736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdb5ca35665545119e4ca1f379c37a73', 'request_id': 'req-fd640b0e-7842-4bd1-b08d-f63e423f9736'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:36.375336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fd640b0e-7842-4bd1-b08d-f63e423f9736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdb5ca35665545119e4ca1f379c37a73', 'request_id': 'req-fd640b0e-7842-4bd1-b08d-f63e423f9736'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:36.385649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fd640b0e-7842-4bd1-b08d-f63e423f9736 admin admin] The token cache is being invalidate because project bdb5ca35665545119e4ca1f379c37a73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:03:36.385960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fd640b0e-7842-4bd1-b08d-f63e423f9736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:36.386582 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-fd640b0e-7842-4bd1-b08d-f63e423f9736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:36.388079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1255/2469] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:03:36 2022] DELETE /identity/v3/projects/bdb5ca35665545119e4ca1f379c37a73 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:36.393122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccf8a5-ecd5-4c54-833e-bccccc55f7fd None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:36.393409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccf8a5-ecd5-4c54-833e-bccccc55f7fd None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:36.393685 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ccf8a5-ecd5-4c54-833e-bccccc55f7fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:36.419658 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e1ccf8a5-ecd5-4c54-833e-bccccc55f7fd None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:36.420874 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e1ccf8a5-ecd5-4c54-833e-bccccc55f7fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:36.444835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:36.478589 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1215/2470] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:36.486773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5548a-e3cc-492b-bb75-671166faeb9c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:36.487051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5548a-e3cc-492b-bb75-671166faeb9c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:36.487384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5548a-e3cc-492b-bb75-671166faeb9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:36.508163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5ff5548a-e3cc-492b-bb75-671166faeb9c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:36.509367 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5ff5548a-e3cc-492b-bb75-671166faeb9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:36.517160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1256/2471] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:36.521630 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccf8a5-ecd5-4c54-833e-bccccc55f7fd None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:36.531839 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ccf8a5-ecd5-4c54-833e-bccccc55f7fd None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:36.536645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31432e-51db-4072-a297-6318b128505c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:36.537236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31432e-51db-4072-a297-6318b128505c admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:36.537510 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31432e-51db-4072-a297-6318b128505c admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:36.537826 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31432e-51db-4072-a297-6318b128505c admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:36.538466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31432e-51db-4072-a297-6318b128505c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:03:36.538764 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31432e-51db-4072-a297-6318b128505c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:36.540432 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31432e-51db-4072-a297-6318b128505c admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:36.551675 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1216/2472] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:03:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:03:36.557693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5548a-e3cc-492b-bb75-671166faeb9c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:36.567022 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5548a-e3cc-492b-bb75-671166faeb9c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:36.572443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d58fe2-3541-4e7e-a60e-111216fd59a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:36.573045 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d58fe2-3541-4e7e-a60e-111216fd59a7 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:36.573326 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d58fe2-3541-4e7e-a60e-111216fd59a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:36.573599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d58fe2-3541-4e7e-a60e-111216fd59a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:36.574427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d58fe2-3541-4e7e-a60e-111216fd59a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:36.576354 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d58fe2-3541-4e7e-a60e-111216fd59a7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:36.742625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1257/2473] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:03:36 2022] POST /identity/v3/projects => generated 363 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:36.746901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31432e-51db-4072-a297-6318b128505c admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:36.757324 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31432e-51db-4072-a297-6318b128505c admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:36.811550 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22bdd2-475d-4ccc-9e0a-5f06b65049bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:36.812147 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22bdd2-475d-4ccc-9e0a-5f06b65049bd admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:36.812424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22bdd2-475d-4ccc-9e0a-5f06b65049bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:36.812690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22bdd2-475d-4ccc-9e0a-5f06b65049bd admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:36.813307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22bdd2-475d-4ccc-9e0a-5f06b65049bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:36.815230 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22bdd2-475d-4ccc-9e0a-5f06b65049bd admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:36.894457 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1217/2474] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:03:36 2022] POST /identity/v3/users => generated 282 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:03:36.898588 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d58fe2-3541-4e7e-a60e-111216fd59a7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:36.908659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d58fe2-3541-4e7e-a60e-111216fd59a7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:36.913530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca68fe-4ea3-4876-8196-5caada291cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:36.914116 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca68fe-4ea3-4876-8196-5caada291cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:36.917735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca68fe-4ea3-4876-8196-5caada291cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:36.917735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca68fe-4ea3-4876-8196-5caada291cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:36.917735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca68fe-4ea3-4876-8196-5caada291cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:36.917735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca68fe-4ea3-4876-8196-5caada291cb2 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:36.934588 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1258/2475] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:36.938741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22bdd2-475d-4ccc-9e0a-5f06b65049bd admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:36.950199 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22bdd2-475d-4ccc-9e0a-5f06b65049bd admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:36.954967 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694ae3b-bf7a-4ab1-a5af-a106be7d0d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:36.955563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694ae3b-bf7a-4ab1-a5af-a106be7d0d54 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:36.955837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694ae3b-bf7a-4ab1-a5af-a106be7d0d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:36.956110 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694ae3b-bf7a-4ab1-a5af-a106be7d0d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:36.956735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694ae3b-bf7a-4ab1-a5af-a106be7d0d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:36.958335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694ae3b-bf7a-4ab1-a5af-a106be7d0d54 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:36.971663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1218/2476] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:36.978777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca68fe-4ea3-4876-8196-5caada291cb2 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:36.988828 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca68fe-4ea3-4876-8196-5caada291cb2 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:36.994524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c5c0d-d24f-4d36-b54f-d5f5980aa355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:36.995191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c5c0d-d24f-4d36-b54f-d5f5980aa355 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:36.995583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c5c0d-d24f-4d36-b54f-d5f5980aa355 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:36.996028 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c5c0d-d24f-4d36-b54f-d5f5980aa355 admin admin] PATH_INFO: `/v3/projects/d3e76cdab5a0495d9b753444e19d29cc/users/b5b8d1e8c7c34f89b452b307eeb7f94a/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:37.015273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c5c0d-d24f-4d36-b54f-d5f5980aa355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e76cdab5a0495d9b753444e19d29cc, user_id=b5b8d1e8c7c34f89b452b307eeb7f94a, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:37.018059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c5c0d-d24f-4d36-b54f-d5f5980aa355 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:37.082865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1259/2477] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:36 2022] PUT /identity/v3/projects/d3e76cdab5a0495d9b753444e19d29cc/users/b5b8d1e8c7c34f89b452b307eeb7f94a/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:37.094676 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2f9f6-8a94-47b3-a419-e1dc3d5ccede None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:37.095114 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2f9f6-8a94-47b3-a419-e1dc3d5ccede None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:37.095446 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2f9f6-8a94-47b3-a419-e1dc3d5ccede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:37.126248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-41d2f9f6-8a94-47b3-a419-e1dc3d5ccede None None] MFA Rules not processed for user `b5b8d1e8c7c34f89b452b307eeb7f94a`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:37.127461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-41d2f9f6-8a94-47b3-a419-e1dc3d5ccede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:37.201508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1219/2478] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:37.206069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c5c0d-d24f-4d36-b54f-d5f5980aa355 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:37.252457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:37.270792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c5c0d-d24f-4d36-b54f-d5f5980aa355 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:37.276323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b2aea-ef3f-4c30-a16e-c26139ab2780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:37.276921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b2aea-ef3f-4c30-a16e-c26139ab2780 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:37.277196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b2aea-ef3f-4c30-a16e-c26139ab2780 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:37.277464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6b2aea-ef3f-4c30-a16e-c26139ab2780 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:37.278122 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b2aea-ef3f-4c30-a16e-c26139ab2780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:37.280062 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6b2aea-ef3f-4c30-a16e-c26139ab2780 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:37.388465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1260/2479] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:03:37 2022] POST /identity/v3/projects => generated 365 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:37.393445 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2f9f6-8a94-47b3-a419-e1dc3d5ccede None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:37.404999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2f9f6-8a94-47b3-a419-e1dc3d5ccede None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:37.425782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:37.461704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfdc7c-4d46-4754-a859-6f6552a2624f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:37.462336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfdc7c-4d46-4754-a859-6f6552a2624f admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:37.462613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfdc7c-4d46-4754-a859-6f6552a2624f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:37.462885 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfdc7c-4d46-4754-a859-6f6552a2624f admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:37.463507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfdc7c-4d46-4754-a859-6f6552a2624f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:37.465415 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfdc7c-4d46-4754-a859-6f6552a2624f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:37.632648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1220/2480] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:03:37 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:03:37.636562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b2aea-ef3f-4c30-a16e-c26139ab2780 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:37.646630 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6b2aea-ef3f-4c30-a16e-c26139ab2780 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:37.651885 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e87ca-6ffe-4560-9236-4831e69b4899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:37.652477 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e87ca-6ffe-4560-9236-4831e69b4899 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:37.652750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e87ca-6ffe-4560-9236-4831e69b4899 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:37.653017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e87ca-6ffe-4560-9236-4831e69b4899 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:37.653673 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e87ca-6ffe-4560-9236-4831e69b4899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:37.655282 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e87ca-6ffe-4560-9236-4831e69b4899 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:37.665602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1261/2481] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:37.669398 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfdc7c-4d46-4754-a859-6f6552a2624f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:37.681983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfdc7c-4d46-4754-a859-6f6552a2624f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:37.687121 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e248d2-28cd-43a5-a4f0-9d838b7ff333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:37.687810 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e248d2-28cd-43a5-a4f0-9d838b7ff333 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:37.688114 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e248d2-28cd-43a5-a4f0-9d838b7ff333 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:37.688392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e248d2-28cd-43a5-a4f0-9d838b7ff333 admin admin] PATH_INFO: `/v3/projects/6a1e4e57ee8b4143b7a997431df1fd68/users/725bc362c750421abdc942758bd4ebb1/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:37.704232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e248d2-28cd-43a5-a4f0-9d838b7ff333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a1e4e57ee8b4143b7a997431df1fd68, user_id=725bc362c750421abdc942758bd4ebb1, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:37.706937 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e248d2-28cd-43a5-a4f0-9d838b7ff333 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:37.794494 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1221/2482] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:37 2022] PUT /identity/v3/projects/6a1e4e57ee8b4143b7a997431df1fd68/users/725bc362c750421abdc942758bd4ebb1/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:37.800294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e87ca-6ffe-4560-9236-4831e69b4899 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:37.808525 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e87ca-6ffe-4560-9236-4831e69b4899 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:37.865049 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7832756-16d5-4cdb-8117-bbcc29e5738c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:37.865654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7832756-16d5-4cdb-8117-bbcc29e5738c admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:37.865998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7832756-16d5-4cdb-8117-bbcc29e5738c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:37.866279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7832756-16d5-4cdb-8117-bbcc29e5738c admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:37.866909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7832756-16d5-4cdb-8117-bbcc29e5738c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:37.868503 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7832756-16d5-4cdb-8117-bbcc29e5738c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:37.879032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1262/2483] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:37.883713 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e248d2-28cd-43a5-a4f0-9d838b7ff333 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:37.894958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e248d2-28cd-43a5-a4f0-9d838b7ff333 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:37.905859 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26af73-bcff-4f56-a570-01d953701331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:37.905859 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26af73-bcff-4f56-a570-01d953701331 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:37.905859 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26af73-bcff-4f56-a570-01d953701331 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:37.905859 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26af73-bcff-4f56-a570-01d953701331 admin admin] PATH_INFO: `/v3/projects/6a1e4e57ee8b4143b7a997431df1fd68/users/725bc362c750421abdc942758bd4ebb1/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:37.906425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26af73-bcff-4f56-a570-01d953701331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a1e4e57ee8b4143b7a997431df1fd68, user_id=725bc362c750421abdc942758bd4ebb1, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:37.909212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26af73-bcff-4f56-a570-01d953701331 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:37.919845 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1222/2484] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:37 2022] PUT /identity/v3/projects/6a1e4e57ee8b4143b7a997431df1fd68/users/725bc362c750421abdc942758bd4ebb1/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:03:37.930050 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b299fb-7995-419d-ad96-aec11ffcd222 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:37.930334 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b299fb-7995-419d-ad96-aec11ffcd222 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:37.930603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b299fb-7995-419d-ad96-aec11ffcd222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:37.960172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-57b299fb-7995-419d-ad96-aec11ffcd222 None None] MFA Rules not processed for user `725bc362c750421abdc942758bd4ebb1`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:37.961635 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-57b299fb-7995-419d-ad96-aec11ffcd222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:38.035930 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1263/2485] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:37 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:38.056255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26af73-bcff-4f56-a570-01d953701331 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:38.058110 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6a26af73-bcff-4f56-a570-01d953701331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:38.123941 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26af73-bcff-4f56-a570-01d953701331 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:38.127731 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765cc193-ebd9-4520-a5b4-a45fcf259ab7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:38.128332 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cc193-ebd9-4520-a5b4-a45fcf259ab7 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:38.128604 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cc193-ebd9-4520-a5b4-a45fcf259ab7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:38.128870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765cc193-ebd9-4520-a5b4-a45fcf259ab7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:38.138879 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765cc193-ebd9-4520-a5b4-a45fcf259ab7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:38.140233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765cc193-ebd9-4520-a5b4-a45fcf259ab7 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:38.150438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1223/2486] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:38.405051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b299fb-7995-419d-ad96-aec11ffcd222 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:38.414948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b299fb-7995-419d-ad96-aec11ffcd222 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:38.418974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31b15d-7765-4aea-abbb-c078e9fc11b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:38.419593 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31b15d-7765-4aea-abbb-c078e9fc11b8 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:38.419875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31b15d-7765-4aea-abbb-c078e9fc11b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:38.420152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f31b15d-7765-4aea-abbb-c078e9fc11b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:38.455175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:38.478323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31b15d-7765-4aea-abbb-c078e9fc11b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:38.479743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f31b15d-7765-4aea-abbb-c078e9fc11b8 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:38.488610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1264/2487] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:38 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:48.221375 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765cc193-ebd9-4520-a5b4-a45fcf259ab7 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:48.222835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-765cc193-ebd9-4520-a5b4-a45fcf259ab7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:48.312369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:48.316618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765cc193-ebd9-4520-a5b4-a45fcf259ab7 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:48.322710 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb71ea6c-034b-4f51-979e-6fd13a6d26a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03bc280d0395479981b7c7b08c96001b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:48.323313 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71ea6c-034b-4f51-979e-6fd13a6d26a8 service placement] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:48.323591 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71ea6c-034b-4f51-979e-6fd13a6d26a8 service placement] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:48.323866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb71ea6c-034b-4f51-979e-6fd13a6d26a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:48.325579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bb71ea6c-034b-4f51-979e-6fd13a6d26a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:48.337488 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71ea6c-034b-4f51-979e-6fd13a6d26a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:48.339152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb71ea6c-034b-4f51-979e-6fd13a6d26a8 service placement] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:48.349312 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1224/2488] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:49.010839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31b15d-7765-4aea-abbb-c078e9fc11b8 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:49.012260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4f31b15d-7765-4aea-abbb-c078e9fc11b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:49.079199 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f31b15d-7765-4aea-abbb-c078e9fc11b8 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:49.083809 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293f99e-12a9-4dc5-a58e-2f69aa9e4a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:49.100349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293f99e-12a9-4dc5-a58e-2f69aa9e4a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:49.100349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293f99e-12a9-4dc5-a58e-2f69aa9e4a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:49.100349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293f99e-12a9-4dc5-a58e-2f69aa9e4a6d admin admin] PATH_INFO: `/v3/users/059ab2513b834e16acae42b1fcf5b250` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:49.100349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293f99e-12a9-4dc5-a58e-2f69aa9e4a6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=059ab2513b834e16acae42b1fcf5b250)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:49.100349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293f99e-12a9-4dc5-a58e-2f69aa9e4a6d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:49.400459 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8293f99e-12a9-4dc5-a58e-2f69aa9e4a6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '059ab2513b834e16acae42b1fcf5b250'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:49.512629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8293f99e-12a9-4dc5-a58e-2f69aa9e4a6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '059ab2513b834e16acae42b1fcf5b250'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:49.527091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8293f99e-12a9-4dc5-a58e-2f69aa9e4a6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '059ab2513b834e16acae42b1fcf5b250'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:49.528909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8293f99e-12a9-4dc5-a58e-2f69aa9e4a6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '059ab2513b834e16acae42b1fcf5b250'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:49.529233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-8293f99e-12a9-4dc5-a58e-2f69aa9e4a6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '059ab2513b834e16acae42b1fcf5b250'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:49.542577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1265/2489] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:03:49 2022] DELETE /identity/v3/users/059ab2513b834e16acae42b1fcf5b250 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:03:49.547140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4f80b-2cc2-496c-9127-97610be64303 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:49.547631 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4f80b-2cc2-496c-9127-97610be64303 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:49.548455 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4f80b-2cc2-496c-9127-97610be64303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:49.569674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-05d4f80b-2cc2-496c-9127-97610be64303 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:49.571734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-05d4f80b-2cc2-496c-9127-97610be64303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:49.629534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1225/2490] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:03:49.640139 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293f99e-12a9-4dc5-a58e-2f69aa9e4a6d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:49.641508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8293f99e-12a9-4dc5-a58e-2f69aa9e4a6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:49.698171 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293f99e-12a9-4dc5-a58e-2f69aa9e4a6d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:49.701679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d683a6b-f785-4f7c-817f-440c3e15e41f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:49.702290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d683a6b-f785-4f7c-817f-440c3e15e41f service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:49.702563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d683a6b-f785-4f7c-817f-440c3e15e41f service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:49.702830 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d683a6b-f785-4f7c-817f-440c3e15e41f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:49.712374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d683a6b-f785-4f7c-817f-440c3e15e41f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:49.713770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d683a6b-f785-4f7c-817f-440c3e15e41f service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:49.717674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:49.723679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1266/2491] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:50.176469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4f80b-2cc2-496c-9127-97610be64303 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:50.180857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-05d4f80b-2cc2-496c-9127-97610be64303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:50.191239 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4f80b-2cc2-496c-9127-97610be64303 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:50.196026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:50.196779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:50.197056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:50.197322 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] PATH_INFO: `/v3/projects/738ff63fc1774138abbbe2c91594de0a` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:50.198452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=738ff63fc1774138abbbe2c91594de0a)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:50.200409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:50.364301 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '738ff63fc1774138abbbe2c91594de0a', 'request_id': 'req-aaf5382d-a5d6-4e8d-b661-f70222effd23'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:50.370109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '738ff63fc1774138abbbe2c91594de0a', 'request_id': 'req-aaf5382d-a5d6-4e8d-b661-f70222effd23'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:50.376296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '738ff63fc1774138abbbe2c91594de0a', 'request_id': 'req-aaf5382d-a5d6-4e8d-b661-f70222effd23'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:50.466550 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] The token cache is being invalidate because project 738ff63fc1774138abbbe2c91594de0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:03:50.466763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:50.467385 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:50.468857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1226/2492] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:03:50 2022] DELETE /identity/v3/projects/738ff63fc1774138abbbe2c91594de0a => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:50.472921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d683a6b-f785-4f7c-817f-440c3e15e41f service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:50.483764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d683a6b-f785-4f7c-817f-440c3e15e41f service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:50.541539 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d484c93-82b4-46f4-903d-e6a5fff2af46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:50.542340 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d484c93-82b4-46f4-903d-e6a5fff2af46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:50.542610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d484c93-82b4-46f4-903d-e6a5fff2af46 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:50.542877 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d484c93-82b4-46f4-903d-e6a5fff2af46 admin admin] PATH_INFO: `/v3/users/63885726ab5044e99e755b0cbb3e05ac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:50.543980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d484c93-82b4-46f4-903d-e6a5fff2af46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63885726ab5044e99e755b0cbb3e05ac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:50.546021 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d484c93-82b4-46f4-903d-e6a5fff2af46 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:50.800333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7d484c93-82b4-46f4-903d-e6a5fff2af46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63885726ab5044e99e755b0cbb3e05ac'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:50.857894 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:50.859369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:50.872092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5382d-a5d6-4e8d-b661-f70222effd23 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:50.876697 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9e51a-908a-4179-91a6-ddc09f8d2fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:50.877324 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9e51a-908a-4179-91a6-ddc09f8d2fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:50.877445 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7d484c93-82b4-46f4-903d-e6a5fff2af46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63885726ab5044e99e755b0cbb3e05ac'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:50.877601 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9e51a-908a-4179-91a6-ddc09f8d2fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:50.877903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad9e51a-908a-4179-91a6-ddc09f8d2fc9 admin admin] PATH_INFO: `/v3/users/b5b8d1e8c7c34f89b452b307eeb7f94a` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:50.879059 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9e51a-908a-4179-91a6-ddc09f8d2fc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5b8d1e8c7c34f89b452b307eeb7f94a)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:50.880978 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad9e51a-908a-4179-91a6-ddc09f8d2fc9 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:50.891536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7d484c93-82b4-46f4-903d-e6a5fff2af46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63885726ab5044e99e755b0cbb3e05ac'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:50.892290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7d484c93-82b4-46f4-903d-e6a5fff2af46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63885726ab5044e99e755b0cbb3e05ac'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:50.895780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7d484c93-82b4-46f4-903d-e6a5fff2af46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63885726ab5044e99e755b0cbb3e05ac'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:50.906171 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1267/2493] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:03:50 2022] DELETE /identity/v3/users/63885726ab5044e99e755b0cbb3e05ac => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:51.063700 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6ad9e51a-908a-4179-91a6-ddc09f8d2fc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5b8d1e8c7c34f89b452b307eeb7f94a'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:51.076585 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6ad9e51a-908a-4179-91a6-ddc09f8d2fc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5b8d1e8c7c34f89b452b307eeb7f94a'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:51.079222 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:51.129700 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6ad9e51a-908a-4179-91a6-ddc09f8d2fc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5b8d1e8c7c34f89b452b307eeb7f94a'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:51.130593 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6ad9e51a-908a-4179-91a6-ddc09f8d2fc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5b8d1e8c7c34f89b452b307eeb7f94a'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:51.138840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6ad9e51a-908a-4179-91a6-ddc09f8d2fc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5b8d1e8c7c34f89b452b307eeb7f94a'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:51.145948 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1227/2494] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:03:50 2022] DELETE /identity/v3/users/b5b8d1e8c7c34f89b452b307eeb7f94a => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:51.156167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386c133-9234-453f-9811-a55bc831d6bc None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:51.156519 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386c133-9234-453f-9811-a55bc831d6bc None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:51.156790 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386c133-9234-453f-9811-a55bc831d6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:51.175966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8386c133-9234-453f-9811-a55bc831d6bc None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:51.177172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8386c133-9234-453f-9811-a55bc831d6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:51.237546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1268/2495] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:51.247765 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9e51a-908a-4179-91a6-ddc09f8d2fc9 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:51.249183 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6ad9e51a-908a-4179-91a6-ddc09f8d2fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:51.307335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad9e51a-908a-4179-91a6-ddc09f8d2fc9 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:51.311080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700bce6c-eee4-4ffd-98cc-589701bf2cc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:51.311743 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bce6c-eee4-4ffd-98cc-589701bf2cc0 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:51.312018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bce6c-eee4-4ffd-98cc-589701bf2cc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:51.312283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700bce6c-eee4-4ffd-98cc-589701bf2cc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:51.322306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bce6c-eee4-4ffd-98cc-589701bf2cc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:51.323950 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700bce6c-eee4-4ffd-98cc-589701bf2cc0 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:51.326902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386c133-9234-453f-9811-a55bc831d6bc None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:51.328369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8386c133-9234-453f-9811-a55bc831d6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:51.334647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1228/2496] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:51.354497 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386c133-9234-453f-9811-a55bc831d6bc None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:51.359563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc534ab-6884-4060-8bf2-f06f14924f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:51.360272 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc534ab-6884-4060-8bf2-f06f14924f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:51.360617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc534ab-6884-4060-8bf2-f06f14924f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:51.360957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc534ab-6884-4060-8bf2-f06f14924f6d admin admin] PATH_INFO: `/v3/projects/9436b434ca8c437eb3a87b79dc1718f1` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:51.362276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc534ab-6884-4060-8bf2-f06f14924f6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9436b434ca8c437eb3a87b79dc1718f1)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:51.364247 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc534ab-6884-4060-8bf2-f06f14924f6d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:51.377476 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:51.473417 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ddc534ab-6884-4060-8bf2-f06f14924f6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9436b434ca8c437eb3a87b79dc1718f1', 'request_id': 'req-ddc534ab-6884-4060-8bf2-f06f14924f6d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:51.569187 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ddc534ab-6884-4060-8bf2-f06f14924f6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9436b434ca8c437eb3a87b79dc1718f1', 'request_id': 'req-ddc534ab-6884-4060-8bf2-f06f14924f6d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:51.576161 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ddc534ab-6884-4060-8bf2-f06f14924f6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9436b434ca8c437eb3a87b79dc1718f1', 'request_id': 'req-ddc534ab-6884-4060-8bf2-f06f14924f6d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:51.587595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ddc534ab-6884-4060-8bf2-f06f14924f6d admin admin] The token cache is being invalidate because project 9436b434ca8c437eb3a87b79dc1718f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:03:51.587885 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ddc534ab-6884-4060-8bf2-f06f14924f6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:51.594131 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ddc534ab-6884-4060-8bf2-f06f14924f6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:51.594131 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1269/2497] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:03:51 2022] DELETE /identity/v3/projects/9436b434ca8c437eb3a87b79dc1718f1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:51.595123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdbe18-96e3-4037-968f-eae5b1a14b7c None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:51.595416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdbe18-96e3-4037-968f-eae5b1a14b7c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:51.595690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdbe18-96e3-4037-968f-eae5b1a14b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:51.617203 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-d8bdbe18-96e3-4037-968f-eae5b1a14b7c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:51.618489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d8bdbe18-96e3-4037-968f-eae5b1a14b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:51.678334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1229/2498] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:03:51.686488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaf1b3-2d88-49aa-a96b-d7b3924c5772 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:51.686770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaf1b3-2d88-49aa-a96b-d7b3924c5772 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:51.687052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aaf1b3-2d88-49aa-a96b-d7b3924c5772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:51.707498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e8aaf1b3-2d88-49aa-a96b-d7b3924c5772 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:51.708791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e8aaf1b3-2d88-49aa-a96b-d7b3924c5772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:51.717784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1270/2499] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:51.722214 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdbe18-96e3-4037-968f-eae5b1a14b7c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:51.732354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdbe18-96e3-4037-968f-eae5b1a14b7c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:51.737017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dd528-a723-4d3d-8fc0-25b66d952282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:51.737669 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dd528-a723-4d3d-8fc0-25b66d952282 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:51.737987 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dd528-a723-4d3d-8fc0-25b66d952282 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:51.738288 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19dd528-a723-4d3d-8fc0-25b66d952282 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:51.738929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19dd528-a723-4d3d-8fc0-25b66d952282 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:03:51.739284 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19dd528-a723-4d3d-8fc0-25b66d952282 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:51.740883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19dd528-a723-4d3d-8fc0-25b66d952282 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:51.752425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1230/2500] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:03:51.756626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aaf1b3-2d88-49aa-a96b-d7b3924c5772 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:51.765408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aaf1b3-2d88-49aa-a96b-d7b3924c5772 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:51.770098 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc04b94-90fe-44a6-aaf4-43e741120b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:51.770788 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc04b94-90fe-44a6-aaf4-43e741120b9a admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:51.770950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc04b94-90fe-44a6-aaf4-43e741120b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:51.771228 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc04b94-90fe-44a6-aaf4-43e741120b9a admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:51.771834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc04b94-90fe-44a6-aaf4-43e741120b9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:51.774237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc04b94-90fe-44a6-aaf4-43e741120b9a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:51.795082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dd528-a723-4d3d-8fc0-25b66d952282 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:51.796792 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c19dd528-a723-4d3d-8fc0-25b66d952282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:51.817645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19dd528-a723-4d3d-8fc0-25b66d952282 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:51.823989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:51.824636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:51.824917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:51.825250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] PATH_INFO: `/v3/projects/d3e76cdab5a0495d9b753444e19d29cc` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:51.826410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3e76cdab5a0495d9b753444e19d29cc)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:51.828468 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:51.893396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1271/2501] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:03:51 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:51.895396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc04b94-90fe-44a6-aaf4-43e741120b9a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:51.956904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3e76cdab5a0495d9b753444e19d29cc', 'request_id': 'req-57de6c57-e762-4534-a8de-b379762e4a14'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:51.963068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc04b94-90fe-44a6-aaf4-43e741120b9a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:51.963783 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3e76cdab5a0495d9b753444e19d29cc', 'request_id': 'req-57de6c57-e762-4534-a8de-b379762e4a14'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:51.970696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3e76cdab5a0495d9b753444e19d29cc', 'request_id': 'req-57de6c57-e762-4534-a8de-b379762e4a14'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:52.018194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba62db0-8e0b-44e3-9616-ea8aa9a1a624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:52.018793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba62db0-8e0b-44e3-9616-ea8aa9a1a624 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:52.019099 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba62db0-8e0b-44e3-9616-ea8aa9a1a624 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:52.019367 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba62db0-8e0b-44e3-9616-ea8aa9a1a624 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:52.020005 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba62db0-8e0b-44e3-9616-ea8aa9a1a624 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:52.021982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba62db0-8e0b-44e3-9616-ea8aa9a1a624 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:52.044507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:52.049186 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] The token cache is being invalidate because project d3e76cdab5a0495d9b753444e19d29cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:03:52.049473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:52.050118 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:52.051599 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1231/2502] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:03:51 2022] DELETE /identity/v3/projects/d3e76cdab5a0495d9b753444e19d29cc => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:03:52.055919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:52.057222 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:52.069218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de6c57-e762-4534-a8de-b379762e4a14 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:52.074320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65093d6-34a1-418d-a33a-03b91a46c299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:52.074913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65093d6-34a1-418d-a33a-03b91a46c299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:52.075190 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65093d6-34a1-418d-a33a-03b91a46c299 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:52.075456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65093d6-34a1-418d-a33a-03b91a46c299 admin admin] PATH_INFO: `/v3/users/725bc362c750421abdc942758bd4ebb1` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:52.076579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65093d6-34a1-418d-a33a-03b91a46c299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=725bc362c750421abdc942758bd4ebb1)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:52.078521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65093d6-34a1-418d-a33a-03b91a46c299 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:52.192281 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1272/2503] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:03:51 2022] POST /identity/v3/users => generated 285 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:03:52.196264 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba62db0-8e0b-44e3-9616-ea8aa9a1a624 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:52.198425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5ba62db0-8e0b-44e3-9616-ea8aa9a1a624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:52.209773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba62db0-8e0b-44e3-9616-ea8aa9a1a624 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:52.214705 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d2e62-ace3-4401-9ad1-692fcb28f370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:52.215400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d2e62-ace3-4401-9ad1-692fcb28f370 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:52.215674 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d2e62-ace3-4401-9ad1-692fcb28f370 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:52.215943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d2e62-ace3-4401-9ad1-692fcb28f370 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:52.216601 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d2e62-ace3-4401-9ad1-692fcb28f370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:52.218396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d2e62-ace3-4401-9ad1-692fcb28f370 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:52.220494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:52.229129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1273/2504] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:03:52.232960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d2e62-ace3-4401-9ad1-692fcb28f370 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:52.243133 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d2e62-ace3-4401-9ad1-692fcb28f370 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:52.249213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fdcde-23a0-492a-a39a-23a5d0b621f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:52.249213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fdcde-23a0-492a-a39a-23a5d0b621f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:52.249213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fdcde-23a0-492a-a39a-23a5d0b621f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:52.249213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fdcde-23a0-492a-a39a-23a5d0b621f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:52.249638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fdcde-23a0-492a-a39a-23a5d0b621f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:52.251192 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fdcde-23a0-492a-a39a-23a5d0b621f8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:52.262804 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1274/2505] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:03:52.266702 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fdcde-23a0-492a-a39a-23a5d0b621f8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:52.275833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fdcde-23a0-492a-a39a-23a5d0b621f8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:52.280457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d619c3e-183a-4433-bf54-d7ef582a6f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:52.281088 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619c3e-183a-4433-bf54-d7ef582a6f2a admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:52.281358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619c3e-183a-4433-bf54-d7ef582a6f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:52.281627 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619c3e-183a-4433-bf54-d7ef582a6f2a admin admin] PATH_INFO: `/v3/projects/3a3ebae410574adb80125acea574f8ac/users/338e3581d451408388d5189af351d83e/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:52.296040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d619c3e-183a-4433-bf54-d7ef582a6f2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a3ebae410574adb80125acea574f8ac, user_id=338e3581d451408388d5189af351d83e, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:52.298688 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d619c3e-183a-4433-bf54-d7ef582a6f2a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:52.309210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d65093d6-34a1-418d-a33a-03b91a46c299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '725bc362c750421abdc942758bd4ebb1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:52.320863 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d65093d6-34a1-418d-a33a-03b91a46c299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '725bc362c750421abdc942758bd4ebb1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:52.392566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1275/2506] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:52 2022] PUT /identity/v3/projects/3a3ebae410574adb80125acea574f8ac/users/338e3581d451408388d5189af351d83e/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:03:52.402801 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf3b7b-9de3-43bb-9e37-f917713ec419 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:52.403081 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf3b7b-9de3-43bb-9e37-f917713ec419 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:52.403349 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf3b7b-9de3-43bb-9e37-f917713ec419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:52.431263 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-66bf3b7b-9de3-43bb-9e37-f917713ec419 None None] MFA Rules not processed for user `338e3581d451408388d5189af351d83e`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:52.432462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-66bf3b7b-9de3-43bb-9e37-f917713ec419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:52.436918 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d65093d6-34a1-418d-a33a-03b91a46c299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '725bc362c750421abdc942758bd4ebb1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:52.438273 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d65093d6-34a1-418d-a33a-03b91a46c299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '725bc362c750421abdc942758bd4ebb1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:52.445714 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d65093d6-34a1-418d-a33a-03b91a46c299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '725bc362c750421abdc942758bd4ebb1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:52.452167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1232/2507] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:03:52 2022] DELETE /identity/v3/users/725bc362c750421abdc942758bd4ebb1 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:03:52.510769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1276/2508] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:03:52.514992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65093d6-34a1-418d-a33a-03b91a46c299 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:52.516688 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d65093d6-34a1-418d-a33a-03b91a46c299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:52.589830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65093d6-34a1-418d-a33a-03b91a46c299 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:52.594758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541b375-3627-41ed-8c56-198bdf084b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:52.595346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541b375-3627-41ed-8c56-198bdf084b81 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:52.595627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541b375-3627-41ed-8c56-198bdf084b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:52.596038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541b375-3627-41ed-8c56-198bdf084b81 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:52.596595 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541b375-3627-41ed-8c56-198bdf084b81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:52.598627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541b375-3627-41ed-8c56-198bdf084b81 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:52.681147 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1233/2509] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:03:52 2022] POST /identity/v3/projects => generated 369 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:03:52.687053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf3b7b-9de3-43bb-9e37-f917713ec419 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:52.771882 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf3b7b-9de3-43bb-9e37-f917713ec419 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:52.840746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abeff7-f624-4201-b3df-01bac228efbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:52.841281 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abeff7-f624-4201-b3df-01bac228efbd admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:52.841560 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abeff7-f624-4201-b3df-01bac228efbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:52.841865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abeff7-f624-4201-b3df-01bac228efbd admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:52.842494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abeff7-f624-4201-b3df-01bac228efbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:52.844423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abeff7-f624-4201-b3df-01bac228efbd admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:52.891667 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541b375-3627-41ed-8c56-198bdf084b81 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:52.893467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4541b375-3627-41ed-8c56-198bdf084b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:52.908790 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541b375-3627-41ed-8c56-198bdf084b81 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:52.914213 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574147cc-d900-4b81-a00e-b75167868d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:52.915094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574147cc-d900-4b81-a00e-b75167868d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:52.915416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574147cc-d900-4b81-a00e-b75167868d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:52.915874 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574147cc-d900-4b81-a00e-b75167868d59 admin admin] PATH_INFO: `/v3/projects/6a1e4e57ee8b4143b7a997431df1fd68` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:52.919920 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574147cc-d900-4b81-a00e-b75167868d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a1e4e57ee8b4143b7a997431df1fd68)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:52.920660 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574147cc-d900-4b81-a00e-b75167868d59 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:52.949685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1277/2510] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:03:52 2022] POST /identity/v3/users => generated 291 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:52.953672 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abeff7-f624-4201-b3df-01bac228efbd admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:52.964303 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:52.967171 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abeff7-f624-4201-b3df-01bac228efbd admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:52.972928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93d0cd-13bf-4aec-82b7-5e9adec229e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:52.973537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93d0cd-13bf-4aec-82b7-5e9adec229e0 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:52.973844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93d0cd-13bf-4aec-82b7-5e9adec229e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:52.974129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93d0cd-13bf-4aec-82b7-5e9adec229e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:52.974896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93d0cd-13bf-4aec-82b7-5e9adec229e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:52.977523 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93d0cd-13bf-4aec-82b7-5e9adec229e0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:52.989837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1278/2511] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:52.991069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93d0cd-13bf-4aec-82b7-5e9adec229e0 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:53.003828 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93d0cd-13bf-4aec-82b7-5e9adec229e0 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:53.010952 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830ae39-1cc0-4d8e-871b-392a688d80fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:53.011602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830ae39-1cc0-4d8e-871b-392a688d80fe admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:53.011978 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830ae39-1cc0-4d8e-871b-392a688d80fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:53.012255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6830ae39-1cc0-4d8e-871b-392a688d80fe admin admin] PATH_INFO: `/v3/projects/e4b2cb38c9aa4ed286d4e881ec226afe/users/fc9eada2897c4182b560c54155baeba1/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:53.028800 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830ae39-1cc0-4d8e-871b-392a688d80fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4b2cb38c9aa4ed286d4e881ec226afe, user_id=fc9eada2897c4182b560c54155baeba1, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:53.031486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6830ae39-1cc0-4d8e-871b-392a688d80fe admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:53.044303 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-574147cc-d900-4b81-a00e-b75167868d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a1e4e57ee8b4143b7a997431df1fd68', 'request_id': 'req-574147cc-d900-4b81-a00e-b75167868d59'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:53.045560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:53.050320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-574147cc-d900-4b81-a00e-b75167868d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a1e4e57ee8b4143b7a997431df1fd68', 'request_id': 'req-574147cc-d900-4b81-a00e-b75167868d59'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:53.057401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-574147cc-d900-4b81-a00e-b75167868d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a1e4e57ee8b4143b7a997431df1fd68', 'request_id': 'req-574147cc-d900-4b81-a00e-b75167868d59'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:53.170575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1279/2512] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:52 2022] PUT /identity/v3/projects/e4b2cb38c9aa4ed286d4e881ec226afe/users/fc9eada2897c4182b560c54155baeba1/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:03:53.173574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-574147cc-d900-4b81-a00e-b75167868d59 admin admin] The token cache is being invalidate because project 6a1e4e57ee8b4143b7a997431df1fd68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:03:53.173914 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-574147cc-d900-4b81-a00e-b75167868d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:53.174579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-574147cc-d900-4b81-a00e-b75167868d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:53.176401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830ae39-1cc0-4d8e-871b-392a688d80fe admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:53.178501 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1234/2513] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:03:52 2022] DELETE /identity/v3/projects/6a1e4e57ee8b4143b7a997431df1fd68 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:03:53.178740 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6830ae39-1cc0-4d8e-871b-392a688d80fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:53.183627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd49e6a-40b9-45b1-8aef-8463308ce5ef None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:53.188188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd49e6a-40b9-45b1-8aef-8463308ce5ef None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:53.188188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd49e6a-40b9-45b1-8aef-8463308ce5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:53.208404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-acd49e6a-40b9-45b1-8aef-8463308ce5ef None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:53.209639 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-acd49e6a-40b9-45b1-8aef-8463308ce5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:53.254248 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6830ae39-1cc0-4d8e-871b-392a688d80fe admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:53.260416 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c5f266-f314-4953-b165-41d788d1ac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:53.261166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5f266-f314-4953-b165-41d788d1ac85 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:53.261450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5f266-f314-4953-b165-41d788d1ac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:53.261909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c5f266-f314-4953-b165-41d788d1ac85 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:53.262617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5f266-f314-4953-b165-41d788d1ac85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:53.264226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c5f266-f314-4953-b165-41d788d1ac85 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:53.275491 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1235/2514] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:03:53.281728 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1280/2515] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:53.282795 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd49e6a-40b9-45b1-8aef-8463308ce5ef None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:53.283875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a692f20-d3d3-48ae-bc05-66d5f5ef0d82 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:53.284274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a692f20-d3d3-48ae-bc05-66d5f5ef0d82 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:53.284827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a692f20-d3d3-48ae-bc05-66d5f5ef0d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:53.292450 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd49e6a-40b9-45b1-8aef-8463308ce5ef None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:53.297407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4caeb0c-5725-46b2-a1a1-ff6a35189616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:53.298105 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caeb0c-5725-46b2-a1a1-ff6a35189616 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:53.298424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caeb0c-5725-46b2-a1a1-ff6a35189616 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:53.298678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caeb0c-5725-46b2-a1a1-ff6a35189616 admin admin] PATH_INFO: `/v3/projects/e4b2cb38c9aa4ed286d4e881ec226afe/users/fc9eada2897c4182b560c54155baeba1/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:53.300634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4caeb0c-5725-46b2-a1a1-ff6a35189616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4b2cb38c9aa4ed286d4e881ec226afe, user_id=fc9eada2897c4182b560c54155baeba1, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:53.303328 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4caeb0c-5725-46b2-a1a1-ff6a35189616 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:53.306597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-8a692f20-d3d3-48ae-bc05-66d5f5ef0d82 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:53.307798 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8a692f20-d3d3-48ae-bc05-66d5f5ef0d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:53.314305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1236/2516] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:53 2022] PUT /identity/v3/projects/e4b2cb38c9aa4ed286d4e881ec226afe/users/fc9eada2897c4182b560c54155baeba1/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:53.326437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2932ba-90b8-4f06-956a-e3b181489804 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:53.326724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2932ba-90b8-4f06-956a-e3b181489804 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:53.326997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2932ba-90b8-4f06-956a-e3b181489804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:53.358569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-6c2932ba-90b8-4f06-956a-e3b181489804 None None] MFA Rules not processed for user `fc9eada2897c4182b560c54155baeba1`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:53.360009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6c2932ba-90b8-4f06-956a-e3b181489804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:53.374625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1281/2517] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:03:53.375709 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a692f20-d3d3-48ae-bc05-66d5f5ef0d82 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:53.386549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a692f20-d3d3-48ae-bc05-66d5f5ef0d82 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:53.391406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fdbeb-3067-4f4d-8178-61f3c2260ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:53.392095 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fdbeb-3067-4f4d-8178-61f3c2260ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:53.392441 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fdbeb-3067-4f4d-8178-61f3c2260ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:53.392798 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7fdbeb-3067-4f4d-8178-61f3c2260ac2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:53.393536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7fdbeb-3067-4f4d-8178-61f3c2260ac2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:03:53.393942 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7fdbeb-3067-4f4d-8178-61f3c2260ac2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:53.395723 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7fdbeb-3067-4f4d-8178-61f3c2260ac2 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:53.414195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1282/2518] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 20:03:53.415478 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fdbeb-3067-4f4d-8178-61f3c2260ac2 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:53.433543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7fdbeb-3067-4f4d-8178-61f3c2260ac2 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:53.440294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21e38b-2a87-437a-8d3f-b5b18f84ec12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:53.441248 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21e38b-2a87-437a-8d3f-b5b18f84ec12 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:53.441593 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21e38b-2a87-437a-8d3f-b5b18f84ec12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:53.441971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21e38b-2a87-437a-8d3f-b5b18f84ec12 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:53.442683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21e38b-2a87-437a-8d3f-b5b18f84ec12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:53.445983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd21e38b-2a87-437a-8d3f-b5b18f84ec12 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:53.454533 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1237/2519] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:03:53.468900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2932ba-90b8-4f06-956a-e3b181489804 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:53.474631 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6c2932ba-90b8-4f06-956a-e3b181489804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:53.550480 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2932ba-90b8-4f06-956a-e3b181489804 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:53.550681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1283/2520] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:03:53 2022] POST /identity/v3/projects => generated 367 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:53.552962 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21e38b-2a87-437a-8d3f-b5b18f84ec12 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:53.616462 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba5865-1ca1-493b-bbde-91d221cd3a9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:53.617584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba5865-1ca1-493b-bbde-91d221cd3a9c service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:53.617901 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba5865-1ca1-493b-bbde-91d221cd3a9c service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:53.618468 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba5865-1ca1-493b-bbde-91d221cd3a9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:53.650668 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21e38b-2a87-437a-8d3f-b5b18f84ec12 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:53.655754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42632eec-c340-44b7-b977-1d164036d89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:53.656468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42632eec-c340-44b7-b977-1d164036d89b admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:53.656844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42632eec-c340-44b7-b977-1d164036d89b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:53.657265 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42632eec-c340-44b7-b977-1d164036d89b admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:53.658031 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42632eec-c340-44b7-b977-1d164036d89b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:53.660214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42632eec-c340-44b7-b977-1d164036d89b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:53.703127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba5865-1ca1-493b-bbde-91d221cd3a9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:53.706041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba5865-1ca1-493b-bbde-91d221cd3a9c service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:53.729929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1238/2521] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:03:53.754704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1284/2522] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:03:53 2022] POST /identity/v3/users => generated 284 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:53.756400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba5865-1ca1-493b-bbde-91d221cd3a9c service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:53.768159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba5865-1ca1-493b-bbde-91d221cd3a9c service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:53.775107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be626c-4784-4c88-aa58-c84079a74707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:53.776042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be626c-4784-4c88-aa58-c84079a74707 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:53.776336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be626c-4784-4c88-aa58-c84079a74707 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:53.776608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be626c-4784-4c88-aa58-c84079a74707 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:53.777268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be626c-4784-4c88-aa58-c84079a74707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:53.778954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be626c-4784-4c88-aa58-c84079a74707 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:53.781434 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:53.791324 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1239/2523] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:53.795292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42632eec-c340-44b7-b977-1d164036d89b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:53.806284 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42632eec-c340-44b7-b977-1d164036d89b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:53.812120 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419dc5f-86cf-4bf9-8764-51409056094c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:53.812733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419dc5f-86cf-4bf9-8764-51409056094c admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:53.813007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419dc5f-86cf-4bf9-8764-51409056094c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:53.813275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419dc5f-86cf-4bf9-8764-51409056094c admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:53.814100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419dc5f-86cf-4bf9-8764-51409056094c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:53.816162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419dc5f-86cf-4bf9-8764-51409056094c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:53.826835 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1285/2524] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:53.832525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be626c-4784-4c88-aa58-c84079a74707 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:53.842991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be626c-4784-4c88-aa58-c84079a74707 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:53.848581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50586265-37a9-4677-b60c-1e9c7892776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:53.849262 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50586265-37a9-4677-b60c-1e9c7892776b admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:53.849526 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50586265-37a9-4677-b60c-1e9c7892776b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:53.849830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50586265-37a9-4677-b60c-1e9c7892776b admin admin] PATH_INFO: `/v3/projects/7089075664e14f6cbd0ebce942986d9d/users/5744c197c7e347b9a55debca44ec0265/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:53.879343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50586265-37a9-4677-b60c-1e9c7892776b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7089075664e14f6cbd0ebce942986d9d, user_id=5744c197c7e347b9a55debca44ec0265, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:53.882383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50586265-37a9-4677-b60c-1e9c7892776b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:54.042819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1240/2525] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:53 2022] PUT /identity/v3/projects/7089075664e14f6cbd0ebce942986d9d/users/5744c197c7e347b9a55debca44ec0265/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:54.046105 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27744f0d-b632-427c-9f17-a4f587ca03f5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:54.046450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27744f0d-b632-427c-9f17-a4f587ca03f5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:54.046801 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27744f0d-b632-427c-9f17-a4f587ca03f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:54.076844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-27744f0d-b632-427c-9f17-a4f587ca03f5 None None] MFA Rules not processed for user `5744c197c7e347b9a55debca44ec0265`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:54.078085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-27744f0d-b632-427c-9f17-a4f587ca03f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:54.142943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1286/2526] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:54.148056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50586265-37a9-4677-b60c-1e9c7892776b admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:54.236662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50586265-37a9-4677-b60c-1e9c7892776b admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:54.241748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956dd879-1606-43cd-8663-c2fd1ebf2124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:54.242498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dd879-1606-43cd-8663-c2fd1ebf2124 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:54.242847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dd879-1606-43cd-8663-c2fd1ebf2124 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:54.243191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956dd879-1606-43cd-8663-c2fd1ebf2124 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:54.243893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956dd879-1606-43cd-8663-c2fd1ebf2124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:54.245950 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956dd879-1606-43cd-8663-c2fd1ebf2124 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:54.359088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1241/2527] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:03:54 2022] POST /identity/v3/projects => generated 369 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:03:54.362240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27744f0d-b632-427c-9f17-a4f587ca03f5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:54.387280 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27744f0d-b632-427c-9f17-a4f587ca03f5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:54.451446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce6407-9a52-41bc-979b-e93298c65732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:54.452194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce6407-9a52-41bc-979b-e93298c65732 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:54.452540 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce6407-9a52-41bc-979b-e93298c65732 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:54.452878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce6407-9a52-41bc-979b-e93298c65732 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:54.453568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce6407-9a52-41bc-979b-e93298c65732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:54.455555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce6407-9a52-41bc-979b-e93298c65732 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:54.568426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1287/2528] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:03:54 2022] POST /identity/v3/users => generated 291 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:54.570152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956dd879-1606-43cd-8663-c2fd1ebf2124 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:54.580373 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956dd879-1606-43cd-8663-c2fd1ebf2124 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:54.584946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cf76d-2db7-481e-b229-9c4663b858b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:54.586443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cf76d-2db7-481e-b229-9c4663b858b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:54.586726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cf76d-2db7-481e-b229-9c4663b858b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:54.586998 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19cf76d-2db7-481e-b229-9c4663b858b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:54.587630 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19cf76d-2db7-481e-b229-9c4663b858b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:54.589279 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19cf76d-2db7-481e-b229-9c4663b858b7 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:54.599666 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1242/2529] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:54.609669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce6407-9a52-41bc-979b-e93298c65732 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:54.619706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce6407-9a52-41bc-979b-e93298c65732 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:54.624332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63970495-b71d-40b3-a195-d6dbab860108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:54.625066 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63970495-b71d-40b3-a195-d6dbab860108 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:54.625275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63970495-b71d-40b3-a195-d6dbab860108 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:54.625551 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63970495-b71d-40b3-a195-d6dbab860108 admin admin] PATH_INFO: `/v3/projects/0b423d4fd0aa428a9f518f43a83ed5b7/users/b81c2cda6625410eb1a2f2ab191a6693/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:54.642047 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63970495-b71d-40b3-a195-d6dbab860108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b423d4fd0aa428a9f518f43a83ed5b7, user_id=b81c2cda6625410eb1a2f2ab191a6693, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:54.644759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63970495-b71d-40b3-a195-d6dbab860108 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:54.725042 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1288/2530] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:54 2022] PUT /identity/v3/projects/0b423d4fd0aa428a9f518f43a83ed5b7/users/b81c2cda6625410eb1a2f2ab191a6693/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:54.728161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cf76d-2db7-481e-b229-9c4663b858b7 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:54.738152 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19cf76d-2db7-481e-b229-9c4663b858b7 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:54.743636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:54.796982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1900e5-f266-4940-9572-49ed6b00f7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:54.797592 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1900e5-f266-4940-9572-49ed6b00f7ad admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:54.797898 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1900e5-f266-4940-9572-49ed6b00f7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:54.798171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1900e5-f266-4940-9572-49ed6b00f7ad admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:54.798801 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1900e5-f266-4940-9572-49ed6b00f7ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:54.800379 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1900e5-f266-4940-9572-49ed6b00f7ad admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:54.802983 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:54.816827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1243/2531] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:03:54.818012 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63970495-b71d-40b3-a195-d6dbab860108 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:54.829034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63970495-b71d-40b3-a195-d6dbab860108 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:54.833932 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89983199-8a2b-4f9a-87ec-52faa1dc4345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:54.834592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89983199-8a2b-4f9a-87ec-52faa1dc4345 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:54.834869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89983199-8a2b-4f9a-87ec-52faa1dc4345 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:54.835185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89983199-8a2b-4f9a-87ec-52faa1dc4345 admin admin] PATH_INFO: `/v3/projects/0b423d4fd0aa428a9f518f43a83ed5b7/users/b81c2cda6625410eb1a2f2ab191a6693/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:54.837408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89983199-8a2b-4f9a-87ec-52faa1dc4345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b423d4fd0aa428a9f518f43a83ed5b7, user_id=b81c2cda6625410eb1a2f2ab191a6693, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:54.840079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89983199-8a2b-4f9a-87ec-52faa1dc4345 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:54.850654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1289/2532] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:03:54 2022] PUT /identity/v3/projects/0b423d4fd0aa428a9f518f43a83ed5b7/users/b81c2cda6625410eb1a2f2ab191a6693/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:03:54.860926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ac1f7-8ddd-42f6-a64d-d0dd79c462b6 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:54.861236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ac1f7-8ddd-42f6-a64d-d0dd79c462b6 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:54.861488 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ac1f7-8ddd-42f6-a64d-d0dd79c462b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:54.890846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e52ac1f7-8ddd-42f6-a64d-d0dd79c462b6 None None] MFA Rules not processed for user `b81c2cda6625410eb1a2f2ab191a6693`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:54.892153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e52ac1f7-8ddd-42f6-a64d-d0dd79c462b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:54.970026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1244/2533] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:54.978377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89983199-8a2b-4f9a-87ec-52faa1dc4345 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:54.980070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-89983199-8a2b-4f9a-87ec-52faa1dc4345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:55.038820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89983199-8a2b-4f9a-87ec-52faa1dc4345 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:55.041966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75094116-ca2c-403a-87b7-56ef542edc7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:55.042592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75094116-ca2c-403a-87b7-56ef542edc7e service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:55.043113 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75094116-ca2c-403a-87b7-56ef542edc7e service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:55.043703 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75094116-ca2c-403a-87b7-56ef542edc7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:55.053357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75094116-ca2c-403a-87b7-56ef542edc7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:55.055292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75094116-ca2c-403a-87b7-56ef542edc7e service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:55.067981 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1290/2534] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:03:55.189942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ac1f7-8ddd-42f6-a64d-d0dd79c462b6 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:55.200300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ac1f7-8ddd-42f6-a64d-d0dd79c462b6 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:55.204046 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd0875-436e-4fea-9855-e2a3d53ee155 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:55.204663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd0875-436e-4fea-9855-e2a3d53ee155 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:55.206517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd0875-436e-4fea-9855-e2a3d53ee155 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:55.206517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd0875-436e-4fea-9855-e2a3d53ee155 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:55.264562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd0875-436e-4fea-9855-e2a3d53ee155 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:55.266020 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd0875-436e-4fea-9855-e2a3d53ee155 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:55.276014 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1245/2535] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:55.301137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:56.992634 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75094116-ca2c-403a-87b7-56ef542edc7e service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:57.054139 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75094116-ca2c-403a-87b7-56ef542edc7e service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:57.058807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a1b3f-757c-4253-aa2a-3a2f112eb308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:57.059465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a1b3f-757c-4253-aa2a-3a2f112eb308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:57.059743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a1b3f-757c-4253-aa2a-3a2f112eb308 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:57.060016 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a1b3f-757c-4253-aa2a-3a2f112eb308 admin admin] PATH_INFO: `/v3/users/338e3581d451408388d5189af351d83e` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:57.061163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a1b3f-757c-4253-aa2a-3a2f112eb308 admin admin] RBAC: Authorizing `identity:delete_user(user_id=338e3581d451408388d5189af351d83e)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:57.063124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353a1b3f-757c-4253-aa2a-3a2f112eb308 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:57.250795 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-353a1b3f-757c-4253-aa2a-3a2f112eb308 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '338e3581d451408388d5189af351d83e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:57.323323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-353a1b3f-757c-4253-aa2a-3a2f112eb308 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '338e3581d451408388d5189af351d83e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:57.335894 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-353a1b3f-757c-4253-aa2a-3a2f112eb308 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '338e3581d451408388d5189af351d83e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:57.336796 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-353a1b3f-757c-4253-aa2a-3a2f112eb308 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '338e3581d451408388d5189af351d83e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:57.337399 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-353a1b3f-757c-4253-aa2a-3a2f112eb308 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '338e3581d451408388d5189af351d83e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:57.345365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:57.350337 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1291/2536] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:03:56 2022] DELETE /identity/v3/users/338e3581d451408388d5189af351d83e => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:57.355311 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c9b2e-3eea-4cc0-8645-ae9a9ae5a2ac None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:57.356018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c9b2e-3eea-4cc0-8645-ae9a9ae5a2ac None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:57.356018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c9b2e-3eea-4cc0-8645-ae9a9ae5a2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:57.375794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-a33c9b2e-3eea-4cc0-8645-ae9a9ae5a2ac None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:57.377003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a33c9b2e-3eea-4cc0-8645-ae9a9ae5a2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:57.387054 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:57.438607 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1246/2537] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:57.449149 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a1b3f-757c-4253-aa2a-3a2f112eb308 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:57.450614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-353a1b3f-757c-4253-aa2a-3a2f112eb308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:57.509138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a1b3f-757c-4253-aa2a-3a2f112eb308 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:57.512777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a9042-e707-4845-8ae4-479365f42499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:57.513381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a9042-e707-4845-8ae4-479365f42499 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:57.513658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a9042-e707-4845-8ae4-479365f42499 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:57.513968 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a9042-e707-4845-8ae4-479365f42499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:57.522633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a9042-e707-4845-8ae4-479365f42499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:57.528356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a9042-e707-4845-8ae4-479365f42499 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:57.533401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1292/2538] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:03:57.948398 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c9b2e-3eea-4cc0-8645-ae9a9ae5a2ac None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:57.949822 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a33c9b2e-3eea-4cc0-8645-ae9a9ae5a2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:57.962171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c9b2e-3eea-4cc0-8645-ae9a9ae5a2ac None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:57.966750 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:57.967380 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:57.967660 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:57.967932 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] PATH_INFO: `/v3/projects/3a3ebae410574adb80125acea574f8ac` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:57.969043 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a3ebae410574adb80125acea574f8ac)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:57.970982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:58.127981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a3ebae410574adb80125acea574f8ac', 'request_id': 'req-d4062767-1a2d-49c9-9e18-92c81cb0d198'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:58.131871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a3ebae410574adb80125acea574f8ac', 'request_id': 'req-d4062767-1a2d-49c9-9e18-92c81cb0d198'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:58.140498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a3ebae410574adb80125acea574f8ac', 'request_id': 'req-d4062767-1a2d-49c9-9e18-92c81cb0d198'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:58.249865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] The token cache is being invalidate because project 3a3ebae410574adb80125acea574f8ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:03:58.250161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:58.250784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:58.252262 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1247/2539] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:03:57 2022] DELETE /identity/v3/projects/3a3ebae410574adb80125acea574f8ac => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:58.256473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a9042-e707-4845-8ae4-479365f42499 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:58.267625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a9042-e707-4845-8ae4-479365f42499 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:58.324920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406601e-e6fa-4631-8311-4a91c7b747d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:58.325564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406601e-e6fa-4631-8311-4a91c7b747d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:58.325886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406601e-e6fa-4631-8311-4a91c7b747d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:58.326156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406601e-e6fa-4631-8311-4a91c7b747d8 admin admin] PATH_INFO: `/v3/users/fc9eada2897c4182b560c54155baeba1` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:58.327321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f406601e-e6fa-4631-8311-4a91c7b747d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc9eada2897c4182b560c54155baeba1)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:58.329681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f406601e-e6fa-4631-8311-4a91c7b747d8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:58.509389 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f406601e-e6fa-4631-8311-4a91c7b747d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc9eada2897c4182b560c54155baeba1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:58.695368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f406601e-e6fa-4631-8311-4a91c7b747d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc9eada2897c4182b560c54155baeba1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:58.708568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f406601e-e6fa-4631-8311-4a91c7b747d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc9eada2897c4182b560c54155baeba1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:58.709196 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f406601e-e6fa-4631-8311-4a91c7b747d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc9eada2897c4182b560c54155baeba1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:58.709815 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f406601e-e6fa-4631-8311-4a91c7b747d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc9eada2897c4182b560c54155baeba1'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:58.722243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1293/2540] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:03:58 2022] DELETE /identity/v3/users/fc9eada2897c4182b560c54155baeba1 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:03:59.209308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:59.211866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:59.241917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4062767-1a2d-49c9-9e18-92c81cb0d198 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:59.315068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8918f0-6110-4119-a95f-bcafdcb08b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:59.316125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8918f0-6110-4119-a95f-bcafdcb08b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:59.316125 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8918f0-6110-4119-a95f-bcafdcb08b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:59.316366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8918f0-6110-4119-a95f-bcafdcb08b95 admin admin] PATH_INFO: `/v3/projects/e4b2cb38c9aa4ed286d4e881ec226afe` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:59.317401 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8918f0-6110-4119-a95f-bcafdcb08b95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4b2cb38c9aa4ed286d4e881ec226afe)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:59.319516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8918f0-6110-4119-a95f-bcafdcb08b95 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:59.512707 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:59.519593 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5f8918f0-6110-4119-a95f-bcafdcb08b95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4b2cb38c9aa4ed286d4e881ec226afe', 'request_id': 'req-5f8918f0-6110-4119-a95f-bcafdcb08b95'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:59.528403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5f8918f0-6110-4119-a95f-bcafdcb08b95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4b2cb38c9aa4ed286d4e881ec226afe', 'request_id': 'req-5f8918f0-6110-4119-a95f-bcafdcb08b95'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:59.536491 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5f8918f0-6110-4119-a95f-bcafdcb08b95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4b2cb38c9aa4ed286d4e881ec226afe', 'request_id': 'req-5f8918f0-6110-4119-a95f-bcafdcb08b95'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:59.693694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5f8918f0-6110-4119-a95f-bcafdcb08b95 admin admin] The token cache is being invalidate because project e4b2cb38c9aa4ed286d4e881ec226afe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:03:59.693694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5f8918f0-6110-4119-a95f-bcafdcb08b95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:59.693694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5f8918f0-6110-4119-a95f-bcafdcb08b95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:03:59.693694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1248/2541] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:03:59 2022] DELETE /identity/v3/projects/e4b2cb38c9aa4ed286d4e881ec226afe => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:03:59.697755 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe1ebc-fca5-4632-b5ff-c9886f5e52aa None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:59.698254 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe1ebc-fca5-4632-b5ff-c9886f5e52aa None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:59.698372 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fe1ebc-fca5-4632-b5ff-c9886f5e52aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:59.723446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-34fe1ebc-fca5-4632-b5ff-c9886f5e52aa None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:59.723446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-34fe1ebc-fca5-4632-b5ff-c9886f5e52aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:59.732418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:03:59.779526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1294/2542] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:03:59.787647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d200-b4f1-416e-a972-510d513e8443 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:59.788196 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d200-b4f1-416e-a972-510d513e8443 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:59.788577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196d200-b4f1-416e-a972-510d513e8443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:59.807893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-0196d200-b4f1-416e-a972-510d513e8443 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:03:59.809079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0196d200-b4f1-416e-a972-510d513e8443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:03:59.816820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1249/2543] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:03:59.821491 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe1ebc-fca5-4632-b5ff-c9886f5e52aa None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:59.830945 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fe1ebc-fca5-4632-b5ff-c9886f5e52aa None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:59.835824 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a5a44-6df2-4858-8df4-3d626279d13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:59.836757 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5a44-6df2-4858-8df4-3d626279d13d admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:59.837533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5a44-6df2-4858-8df4-3d626279d13d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:59.837833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a5a44-6df2-4858-8df4-3d626279d13d admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:59.838467 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a5a44-6df2-4858-8df4-3d626279d13d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:03:59.838769 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a5a44-6df2-4858-8df4-3d626279d13d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:59.840364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a5a44-6df2-4858-8df4-3d626279d13d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:59.852074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1295/2544] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 20:03:59.856433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196d200-b4f1-416e-a972-510d513e8443 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:59.865597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196d200-b4f1-416e-a972-510d513e8443 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:03:59.870730 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ada6f-ad31-4913-a2be-26e4e229928b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:03:59.871378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ada6f-ad31-4913-a2be-26e4e229928b admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:03:59.871659 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ada6f-ad31-4913-a2be-26e4e229928b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:03:59.871931 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ada6f-ad31-4913-a2be-26e4e229928b admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:03:59.872552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ada6f-ad31-4913-a2be-26e4e229928b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:03:59.876921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ada6f-ad31-4913-a2be-26e4e229928b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:03:59.982066 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1250/2545] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:03:59 2022] POST /identity/v3/projects => generated 387 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:03:59.986332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a5a44-6df2-4858-8df4-3d626279d13d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:03:59.996938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a5a44-6df2-4858-8df4-3d626279d13d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:00.052817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e8413-87a5-4ff1-bafe-723ebb6e84b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:00.054082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e8413-87a5-4ff1-bafe-723ebb6e84b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:00.054082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e8413-87a5-4ff1-bafe-723ebb6e84b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:00.054082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4e8413-87a5-4ff1-bafe-723ebb6e84b0 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:00.054661 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e8413-87a5-4ff1-bafe-723ebb6e84b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:00.057118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4e8413-87a5-4ff1-bafe-723ebb6e84b0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:00.175719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1296/2546] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:03:59 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:00.179829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ada6f-ad31-4913-a2be-26e4e229928b admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:00.190409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ada6f-ad31-4913-a2be-26e4e229928b admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:00.194992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f2cf7-95ab-45fd-aa91-48999fb21f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:00.195572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f2cf7-95ab-45fd-aa91-48999fb21f1d admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:00.196485 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f2cf7-95ab-45fd-aa91-48999fb21f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:00.196749 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f2cf7-95ab-45fd-aa91-48999fb21f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:00.197430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f2cf7-95ab-45fd-aa91-48999fb21f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:00.199150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6f2cf7-95ab-45fd-aa91-48999fb21f1d admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:00.209936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1251/2547] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:00.214483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e8413-87a5-4ff1-bafe-723ebb6e84b0 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:00.224607 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4e8413-87a5-4ff1-bafe-723ebb6e84b0 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:00.229387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d840b7f-d0f4-42f6-ae39-c884efd6c4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:00.230009 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d840b7f-d0f4-42f6-ae39-c884efd6c4e5 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:00.230292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d840b7f-d0f4-42f6-ae39-c884efd6c4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:00.230613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d840b7f-d0f4-42f6-ae39-c884efd6c4e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:00.231264 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d840b7f-d0f4-42f6-ae39-c884efd6c4e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:00.232948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d840b7f-d0f4-42f6-ae39-c884efd6c4e5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:00.244040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1297/2548] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:00.248259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f2cf7-95ab-45fd-aa91-48999fb21f1d admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:00.258728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f2cf7-95ab-45fd-aa91-48999fb21f1d admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:00.263307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e7d2e-1b11-42e9-b8f8-54a0ea523b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:00.263938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e7d2e-1b11-42e9-b8f8-54a0ea523b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:00.264211 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e7d2e-1b11-42e9-b8f8-54a0ea523b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:00.264479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3e7d2e-1b11-42e9-b8f8-54a0ea523b80 admin admin] PATH_INFO: `/v3/projects/5eb8cf8a9df64d1ebd2e9cdd4b066f6b/users/f859b39b91a74a4284c690853ddcaf6b/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:00.279065 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e7d2e-1b11-42e9-b8f8-54a0ea523b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eb8cf8a9df64d1ebd2e9cdd4b066f6b, user_id=f859b39b91a74a4284c690853ddcaf6b, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:00.281875 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3e7d2e-1b11-42e9-b8f8-54a0ea523b80 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:00.386489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1252/2549] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:00 2022] PUT /identity/v3/projects/5eb8cf8a9df64d1ebd2e9cdd4b066f6b/users/f859b39b91a74a4284c690853ddcaf6b/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:00.397831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d742ac-712a-4c4e-bd7d-2d2b7b717a28 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:00.398123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d742ac-712a-4c4e-bd7d-2d2b7b717a28 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:00.398398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d742ac-712a-4c4e-bd7d-2d2b7b717a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:00.427097 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-b0d742ac-712a-4c4e-bd7d-2d2b7b717a28 None None] MFA Rules not processed for user `f859b39b91a74a4284c690853ddcaf6b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:00.428315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b0d742ac-712a-4c4e-bd7d-2d2b7b717a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:00.495828 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1298/2550] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:00.500364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e7d2e-1b11-42e9-b8f8-54a0ea523b80 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:00.561350 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3e7d2e-1b11-42e9-b8f8-54a0ea523b80 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:00.566367 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3544b1-08a4-442b-b99f-8df852ec79b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:00.567129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3544b1-08a4-442b-b99f-8df852ec79b0 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:00.567247 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3544b1-08a4-442b-b99f-8df852ec79b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:00.567519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3544b1-08a4-442b-b99f-8df852ec79b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:00.568174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3544b1-08a4-442b-b99f-8df852ec79b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:00.570166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3544b1-08a4-442b-b99f-8df852ec79b0 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:00.624461 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d742ac-712a-4c4e-bd7d-2d2b7b717a28 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:00.631566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b0d742ac-712a-4c4e-bd7d-2d2b7b717a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:00.639134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d742ac-712a-4c4e-bd7d-2d2b7b717a28 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:00.644129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112e0ff-d7d4-4034-a897-1082c078c71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:00.644841 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112e0ff-d7d4-4034-a897-1082c078c71d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:00.645295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112e0ff-d7d4-4034-a897-1082c078c71d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:00.645570 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0112e0ff-d7d4-4034-a897-1082c078c71d admin admin] PATH_INFO: `/v3/users/5744c197c7e347b9a55debca44ec0265` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:00.646842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112e0ff-d7d4-4034-a897-1082c078c71d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5744c197c7e347b9a55debca44ec0265)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:00.649307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0112e0ff-d7d4-4034-a897-1082c078c71d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:00.660794 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:00.700164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1253/2551] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:00 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:00.704166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3544b1-08a4-442b-b99f-8df852ec79b0 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:00.775945 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:00.785857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3544b1-08a4-442b-b99f-8df852ec79b0 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:00.791209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f0f97-da5f-4d48-a425-b319081c84b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:00.796228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f0f97-da5f-4d48-a425-b319081c84b5 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:00.796228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f0f97-da5f-4d48-a425-b319081c84b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:00.796228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091f0f97-da5f-4d48-a425-b319081c84b5 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:00.796228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f0f97-da5f-4d48-a425-b319081c84b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:00.796228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091f0f97-da5f-4d48-a425-b319081c84b5 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:00.862812 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0112e0ff-d7d4-4034-a897-1082c078c71d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5744c197c7e347b9a55debca44ec0265'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:00.878112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1254/2552] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:00 2022] POST /identity/v3/users => generated 300 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:00.879162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f0f97-da5f-4d48-a425-b319081c84b5 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:00.880732 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-091f0f97-da5f-4d48-a425-b319081c84b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:00.923918 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0112e0ff-d7d4-4034-a897-1082c078c71d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5744c197c7e347b9a55debca44ec0265'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:00.937437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091f0f97-da5f-4d48-a425-b319081c84b5 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:00.943782 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0112e0ff-d7d4-4034-a897-1082c078c71d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5744c197c7e347b9a55debca44ec0265'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:00.943782 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0112e0ff-d7d4-4034-a897-1082c078c71d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5744c197c7e347b9a55debca44ec0265'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:00.943782 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0112e0ff-d7d4-4034-a897-1082c078c71d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5744c197c7e347b9a55debca44ec0265'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:00.944092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cd5732-6124-4edd-8eab-85fbbb8134ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:00.944092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd5732-6124-4edd-8eab-85fbbb8134ef service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:00.944092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd5732-6124-4edd-8eab-85fbbb8134ef service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:00.944092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cd5732-6124-4edd-8eab-85fbbb8134ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:00.944449 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-f4cd5732-6124-4edd-8eab-85fbbb8134ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:00.952605 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1299/2553] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:00 2022] DELETE /identity/v3/users/5744c197c7e347b9a55debca44ec0265 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:00.953216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112e0ff-d7d4-4034-a897-1082c078c71d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:00.955347 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0112e0ff-d7d4-4034-a897-1082c078c71d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:00.957535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd5732-6124-4edd-8eab-85fbbb8134ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:00.959007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cd5732-6124-4edd-8eab-85fbbb8134ef service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:00.969961 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1255/2554] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:00.971308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237a463-c4b2-4e3d-8bea-7b3268955220 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:00.971585 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237a463-c4b2-4e3d-8bea-7b3268955220 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:00.971867 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c237a463-c4b2-4e3d-8bea-7b3268955220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:00.990869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-c237a463-c4b2-4e3d-8bea-7b3268955220 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:00.992127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c237a463-c4b2-4e3d-8bea-7b3268955220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:01.025363 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0112e0ff-d7d4-4034-a897-1082c078c71d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:01.031360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec704fc-e2d9-4223-ada5-71727fe0f11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:01.031977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec704fc-e2d9-4223-ada5-71727fe0f11f admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:01.032261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec704fc-e2d9-4223-ada5-71727fe0f11f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:01.032537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec704fc-e2d9-4223-ada5-71727fe0f11f admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:01.033174 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec704fc-e2d9-4223-ada5-71727fe0f11f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:01.035240 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec704fc-e2d9-4223-ada5-71727fe0f11f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:01.051537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1300/2555] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:01.052570 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec704fc-e2d9-4223-ada5-71727fe0f11f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:01.056655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1256/2556] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:01.064856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec704fc-e2d9-4223-ada5-71727fe0f11f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:01.066874 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237a463-c4b2-4e3d-8bea-7b3268955220 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:01.068416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c237a463-c4b2-4e3d-8bea-7b3268955220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:01.072000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d080e-f013-4c19-a9fb-09416b14a0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:01.072672 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d080e-f013-4c19-a9fb-09416b14a0e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:01.073425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d080e-f013-4c19-a9fb-09416b14a0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:01.073425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918d080e-f013-4c19-a9fb-09416b14a0e8 admin admin] PATH_INFO: `/v3/projects/a99a537b84ac48c3a7d606e0e2f922b9/users/71aaba8342664e1fa061030748a3670a/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:01.091059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d080e-f013-4c19-a9fb-09416b14a0e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a99a537b84ac48c3a7d606e0e2f922b9, user_id=71aaba8342664e1fa061030748a3670a, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:01.093701 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918d080e-f013-4c19-a9fb-09416b14a0e8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:01.128738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c237a463-c4b2-4e3d-8bea-7b3268955220 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:01.132801 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daec7b5-f4f7-4dee-8006-b6e04cfb8c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:01.133443 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daec7b5-f4f7-4dee-8006-b6e04cfb8c79 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:01.133723 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daec7b5-f4f7-4dee-8006-b6e04cfb8c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:01.134044 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daec7b5-f4f7-4dee-8006-b6e04cfb8c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:01.143410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daec7b5-f4f7-4dee-8006-b6e04cfb8c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:01.144882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daec7b5-f4f7-4dee-8006-b6e04cfb8c79 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:01.154869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1257/2557] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:04:01.201571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1301/2558] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:01 2022] PUT /identity/v3/projects/a99a537b84ac48c3a7d606e0e2f922b9/users/71aaba8342664e1fa061030748a3670a/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:01.206907 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daec7b5-f4f7-4dee-8006-b6e04cfb8c79 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:01.220844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daec7b5-f4f7-4dee-8006-b6e04cfb8c79 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:01.248378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:01.280649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca61e2-91bb-447c-8e55-31721b1834f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:01.281174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca61e2-91bb-447c-8e55-31721b1834f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:01.281448 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca61e2-91bb-447c-8e55-31721b1834f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:01.281719 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca61e2-91bb-447c-8e55-31721b1834f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:01.282380 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca61e2-91bb-447c-8e55-31721b1834f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:01.283999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ca61e2-91bb-447c-8e55-31721b1834f6 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:01.297517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1258/2559] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:01.299817 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d080e-f013-4c19-a9fb-09416b14a0e8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:01.311089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918d080e-f013-4c19-a9fb-09416b14a0e8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:01.316368 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced57dbf-5fee-4479-a01c-1fb4817d1fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:01.317031 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced57dbf-5fee-4479-a01c-1fb4817d1fba admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:01.317302 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced57dbf-5fee-4479-a01c-1fb4817d1fba admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:01.317663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced57dbf-5fee-4479-a01c-1fb4817d1fba admin admin] PATH_INFO: `/v3/projects/a99a537b84ac48c3a7d606e0e2f922b9/users/71aaba8342664e1fa061030748a3670a/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:01.319564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced57dbf-5fee-4479-a01c-1fb4817d1fba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a99a537b84ac48c3a7d606e0e2f922b9, user_id=71aaba8342664e1fa061030748a3670a, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:01.322332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced57dbf-5fee-4479-a01c-1fb4817d1fba admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:01.333453 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1302/2560] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:01 2022] PUT /identity/v3/projects/a99a537b84ac48c3a7d606e0e2f922b9/users/71aaba8342664e1fa061030748a3670a/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:01.344286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0cfca-f1e0-48da-8f04-44fec078c696 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:01.344574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0cfca-f1e0-48da-8f04-44fec078c696 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:01.344848 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0cfca-f1e0-48da-8f04-44fec078c696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:01.377216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-18e0cfca-f1e0-48da-8f04-44fec078c696 None None] MFA Rules not processed for user `71aaba8342664e1fa061030748a3670a`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:01.378794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-18e0cfca-f1e0-48da-8f04-44fec078c696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:01.451293 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1259/2561] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:01 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:01.463251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced57dbf-5fee-4479-a01c-1fb4817d1fba admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:01.465061 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ced57dbf-5fee-4479-a01c-1fb4817d1fba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:01.519482 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:01.524698 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced57dbf-5fee-4479-a01c-1fb4817d1fba admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:01.528483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85618c1-899c-4821-be43-476319f4ffc1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:01.529688 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85618c1-899c-4821-be43-476319f4ffc1 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:01.529688 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85618c1-899c-4821-be43-476319f4ffc1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:01.530019 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85618c1-899c-4821-be43-476319f4ffc1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:01.539425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85618c1-899c-4821-be43-476319f4ffc1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:01.541089 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85618c1-899c-4821-be43-476319f4ffc1 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:01.551466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1303/2562] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:01 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:01.582969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0cfca-f1e0-48da-8f04-44fec078c696 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:01.647831 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0cfca-f1e0-48da-8f04-44fec078c696 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:01.653428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c40041-0afd-4d8c-aac3-3e4d89f21710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:01.654206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c40041-0afd-4d8c-aac3-3e4d89f21710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:01.654483 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c40041-0afd-4d8c-aac3-3e4d89f21710 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:01.654758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c40041-0afd-4d8c-aac3-3e4d89f21710 admin admin] PATH_INFO: `/v3/users/f859b39b91a74a4284c690853ddcaf6b` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:01.655910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c40041-0afd-4d8c-aac3-3e4d89f21710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f859b39b91a74a4284c690853ddcaf6b)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:01.657871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c40041-0afd-4d8c-aac3-3e4d89f21710 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:01.816199 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85618c1-899c-4821-be43-476319f4ffc1 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:01.817612 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f85618c1-899c-4821-be43-476319f4ffc1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:01.833086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85618c1-899c-4821-be43-476319f4ffc1 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:01.838283 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:01.838881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:01.839164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:01.839439 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] PATH_INFO: `/v3/projects/7089075664e14f6cbd0ebce942986d9d` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:01.840574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7089075664e14f6cbd0ebce942986d9d)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:01.842638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:01.972117 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-f1c40041-0afd-4d8c-aac3-3e4d89f21710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f859b39b91a74a4284c690853ddcaf6b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:01.973973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:01.985713 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-f1c40041-0afd-4d8c-aac3-3e4d89f21710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f859b39b91a74a4284c690853ddcaf6b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:01.985888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7089075664e14f6cbd0ebce942986d9d', 'request_id': 'req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:02.124209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7089075664e14f6cbd0ebce942986d9d', 'request_id': 'req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:02.135996 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7089075664e14f6cbd0ebce942986d9d', 'request_id': 'req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:02.148029 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] The token cache is being invalidate because project 7089075664e14f6cbd0ebce942986d9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:02.148499 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:02.150786 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:02.158768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1304/2563] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:01 2022] DELETE /identity/v3/projects/7089075664e14f6cbd0ebce942986d9d => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:02.158768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:02.158768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:02.192694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-f1c40041-0afd-4d8c-aac3-3e4d89f21710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f859b39b91a74a4284c690853ddcaf6b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:02.193551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-f1c40041-0afd-4d8c-aac3-3e4d89f21710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f859b39b91a74a4284c690853ddcaf6b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:02.202369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-f1c40041-0afd-4d8c-aac3-3e4d89f21710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f859b39b91a74a4284c690853ddcaf6b'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:02.212654 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1260/2564] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:01 2022] DELETE /identity/v3/users/f859b39b91a74a4284c690853ddcaf6b => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:02.217503 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fa8e7-9d08-4d6e-a116-9b23998ca476 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:02.217943 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fa8e7-9d08-4d6e-a116-9b23998ca476 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:02.218345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993fa8e7-9d08-4d6e-a116-9b23998ca476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:02.243817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-993fa8e7-9d08-4d6e-a116-9b23998ca476 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:02.245530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-993fa8e7-9d08-4d6e-a116-9b23998ca476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:02.245916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca36d5-09b6-40b1-a83b-7d0b445dee5e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:02.321301 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cdb85-157a-4a16-99e2-f128b6560e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:02.322209 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cdb85-157a-4a16-99e2-f128b6560e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:02.322490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cdb85-157a-4a16-99e2-f128b6560e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:02.322901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cdb85-157a-4a16-99e2-f128b6560e68 admin admin] PATH_INFO: `/v3/users/b81c2cda6625410eb1a2f2ab191a6693` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:02.324198 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996cdb85-157a-4a16-99e2-f128b6560e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b81c2cda6625410eb1a2f2ab191a6693)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:02.326312 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996cdb85-157a-4a16-99e2-f128b6560e68 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:02.326790 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1261/2565] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:02.337326 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fa8e7-9d08-4d6e-a116-9b23998ca476 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:02.339430 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-993fa8e7-9d08-4d6e-a116-9b23998ca476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:02.350998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:02.398161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993fa8e7-9d08-4d6e-a116-9b23998ca476 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:02.402361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f609a-8fe4-4b2f-bcbd-144afa4151a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:02.402983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f609a-8fe4-4b2f-bcbd-144afa4151a3 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:02.403256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f609a-8fe4-4b2f-bcbd-144afa4151a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:02.403525 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f609a-8fe4-4b2f-bcbd-144afa4151a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:02.412744 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f609a-8fe4-4b2f-bcbd-144afa4151a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:02.414233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8f609a-8fe4-4b2f-bcbd-144afa4151a3 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:02.424840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1262/2566] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:02.573190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-996cdb85-157a-4a16-99e2-f128b6560e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b81c2cda6625410eb1a2f2ab191a6693'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:02.666293 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-996cdb85-157a-4a16-99e2-f128b6560e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b81c2cda6625410eb1a2f2ab191a6693'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:02.692944 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-996cdb85-157a-4a16-99e2-f128b6560e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b81c2cda6625410eb1a2f2ab191a6693'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:02.694258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-996cdb85-157a-4a16-99e2-f128b6560e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b81c2cda6625410eb1a2f2ab191a6693'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:02.696743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-996cdb85-157a-4a16-99e2-f128b6560e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b81c2cda6625410eb1a2f2ab191a6693'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:02.715974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1305/2567] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:02 2022] DELETE /identity/v3/users/b81c2cda6625410eb1a2f2ab191a6693 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:02.930846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f609a-8fe4-4b2f-bcbd-144afa4151a3 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:02.932358 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4b8f609a-8fe4-4b2f-bcbd-144afa4151a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:02.956857 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f609a-8fe4-4b2f-bcbd-144afa4151a3 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:03.092539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:03.093614 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:03.094135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:03.094643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] PATH_INFO: `/v3/projects/5eb8cf8a9df64d1ebd2e9cdd4b066f6b` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:03.097166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eb8cf8a9df64d1ebd2e9cdd4b066f6b)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:03.101868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:03.117097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:03.212570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eb8cf8a9df64d1ebd2e9cdd4b066f6b', 'request_id': 'req-4ca95c38-3d5c-46a8-b712-743972242899'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.219128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eb8cf8a9df64d1ebd2e9cdd4b066f6b', 'request_id': 'req-4ca95c38-3d5c-46a8-b712-743972242899'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.245055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eb8cf8a9df64d1ebd2e9cdd4b066f6b', 'request_id': 'req-4ca95c38-3d5c-46a8-b712-743972242899'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.331256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] The token cache is being invalidate because project 5eb8cf8a9df64d1ebd2e9cdd4b066f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:03.331552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.332223 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.338263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1263/2568] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:02 2022] DELETE /identity/v3/projects/5eb8cf8a9df64d1ebd2e9cdd4b066f6b => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:03.339236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cdb85-157a-4a16-99e2-f128b6560e68 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:03.340906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-996cdb85-157a-4a16-99e2-f128b6560e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:03.358390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cdb85-157a-4a16-99e2-f128b6560e68 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:03.389041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:03.390712 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:03.446654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02550675-1605-4dfd-b120-d2788131c942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:03.447420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02550675-1605-4dfd-b120-d2788131c942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:03.447701 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02550675-1605-4dfd-b120-d2788131c942 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:03.447980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02550675-1605-4dfd-b120-d2788131c942 admin admin] PATH_INFO: `/v3/users/71aaba8342664e1fa061030748a3670a` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:03.449213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02550675-1605-4dfd-b120-d2788131c942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71aaba8342664e1fa061030748a3670a)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:03.451564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02550675-1605-4dfd-b120-d2788131c942 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:03.475198 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca95c38-3d5c-46a8-b712-743972242899 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:03.482418 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec77dd-2ba0-4d32-9432-dd650891e750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:03.483056 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec77dd-2ba0-4d32-9432-dd650891e750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:03.483385 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec77dd-2ba0-4d32-9432-dd650891e750 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:03.483667 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec77dd-2ba0-4d32-9432-dd650891e750 admin admin] PATH_INFO: `/v3/projects/0b423d4fd0aa428a9f518f43a83ed5b7` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:03.484946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec77dd-2ba0-4d32-9432-dd650891e750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b423d4fd0aa428a9f518f43a83ed5b7)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:03.486940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec77dd-2ba0-4d32-9432-dd650891e750 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:03.652895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-96ec77dd-2ba0-4d32-9432-dd650891e750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b423d4fd0aa428a9f518f43a83ed5b7', 'request_id': 'req-96ec77dd-2ba0-4d32-9432-dd650891e750'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.653094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02550675-1605-4dfd-b120-d2788131c942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71aaba8342664e1fa061030748a3670a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.658469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-96ec77dd-2ba0-4d32-9432-dd650891e750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b423d4fd0aa428a9f518f43a83ed5b7', 'request_id': 'req-96ec77dd-2ba0-4d32-9432-dd650891e750'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.665429 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-96ec77dd-2ba0-4d32-9432-dd650891e750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b423d4fd0aa428a9f518f43a83ed5b7', 'request_id': 'req-96ec77dd-2ba0-4d32-9432-dd650891e750'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.733941 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02550675-1605-4dfd-b120-d2788131c942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71aaba8342664e1fa061030748a3670a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.748752 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02550675-1605-4dfd-b120-d2788131c942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71aaba8342664e1fa061030748a3670a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.749479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02550675-1605-4dfd-b120-d2788131c942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71aaba8342664e1fa061030748a3670a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.750110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-02550675-1605-4dfd-b120-d2788131c942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71aaba8342664e1fa061030748a3670a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.762435 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1306/2569] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:03 2022] DELETE /identity/v3/users/71aaba8342664e1fa061030748a3670a => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:03.827476 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-96ec77dd-2ba0-4d32-9432-dd650891e750 admin admin] The token cache is being invalidate because project 0b423d4fd0aa428a9f518f43a83ed5b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:03.828149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-96ec77dd-2ba0-4d32-9432-dd650891e750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.828694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-96ec77dd-2ba0-4d32-9432-dd650891e750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:03.830605 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1264/2570] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:03 2022] DELETE /identity/v3/projects/0b423d4fd0aa428a9f518f43a83ed5b7 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:03.835508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a3f67-9e5c-42b9-afe5-2a4bd87f3a6d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:03.835802 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a3f67-9e5c-42b9-afe5-2a4bd87f3a6d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:03.836074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a3f67-9e5c-42b9-afe5-2a4bd87f3a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:03.859143 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-629a3f67-9e5c-42b9-afe5-2a4bd87f3a6d None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:03.860555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-629a3f67-9e5c-42b9-afe5-2a4bd87f3a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:03.878750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:03.936072 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1307/2571] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:03.944154 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc10927-08fe-4375-9882-edc95825cdac None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:03.944541 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc10927-08fe-4375-9882-edc95825cdac None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:03.949605 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc10927-08fe-4375-9882-edc95825cdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:03.969778 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-cdc10927-08fe-4375-9882-edc95825cdac None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:03.971233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-cdc10927-08fe-4375-9882-edc95825cdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:03.981571 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1265/2572] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:03.985829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a3f67-9e5c-42b9-afe5-2a4bd87f3a6d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:04.004061 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a3f67-9e5c-42b9-afe5-2a4bd87f3a6d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:04.009134 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a3d10-2215-4cb9-bfe0-3ae7ff1d5dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:04.009751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a3d10-2215-4cb9-bfe0-3ae7ff1d5dbf admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:04.010059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a3d10-2215-4cb9-bfe0-3ae7ff1d5dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:04.010329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a3d10-2215-4cb9-bfe0-3ae7ff1d5dbf admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:04.010979 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a3d10-2215-4cb9-bfe0-3ae7ff1d5dbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:04:04.011286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a3d10-2215-4cb9-bfe0-3ae7ff1d5dbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:04.012886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a3d10-2215-4cb9-bfe0-3ae7ff1d5dbf admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:04.027938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1308/2573] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:04:04.033339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc10927-08fe-4375-9882-edc95825cdac None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:04.045366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc10927-08fe-4375-9882-edc95825cdac None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:04.050892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbed16-179a-47cf-baf2-ced3d9b175f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:04.051507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbed16-179a-47cf-baf2-ced3d9b175f9 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:04.051987 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbed16-179a-47cf-baf2-ced3d9b175f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:04.052269 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdbed16-179a-47cf-baf2-ced3d9b175f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:04.053000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbed16-179a-47cf-baf2-ced3d9b175f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:04.055172 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdbed16-179a-47cf-baf2-ced3d9b175f9 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:04.180386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1266/2574] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:04 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:04.181502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a3d10-2215-4cb9-bfe0-3ae7ff1d5dbf admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:04.192159 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a3d10-2215-4cb9-bfe0-3ae7ff1d5dbf admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:04.245774 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbed16-179a-47cf-baf2-ced3d9b175f9 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:04.248005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-dfdbed16-179a-47cf-baf2-ced3d9b175f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:04.252784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:04.271981 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ec991-cc1a-4a15-9afa-bf7ec73ea7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:04.272606 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ec991-cc1a-4a15-9afa-bf7ec73ea7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:04.272875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ec991-cc1a-4a15-9afa-bf7ec73ea7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:04.273152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ec991-cc1a-4a15-9afa-bf7ec73ea7a6 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:04.274364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ec991-cc1a-4a15-9afa-bf7ec73ea7a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:04.276691 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ec991-cc1a-4a15-9afa-bf7ec73ea7a6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:04.338478 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdbed16-179a-47cf-baf2-ced3d9b175f9 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:04.343142 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b476b-ac27-4d20-965d-3a67a9287f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:04.343768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b476b-ac27-4d20-965d-3a67a9287f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:04.344044 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b476b-ac27-4d20-965d-3a67a9287f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:04.344314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b476b-ac27-4d20-965d-3a67a9287f1f admin admin] PATH_INFO: `/v3/projects/a99a537b84ac48c3a7d606e0e2f922b9` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:04.345481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b476b-ac27-4d20-965d-3a67a9287f1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a99a537b84ac48c3a7d606e0e2f922b9)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:04.347410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b476b-ac27-4d20-965d-3a67a9287f1f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:04.384605 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1309/2575] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:04 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:04.385926 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ec991-cc1a-4a15-9afa-bf7ec73ea7a6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:04.397581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ec991-cc1a-4a15-9afa-bf7ec73ea7a6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:04.403920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bb034-2e29-4e65-b74d-a653205a9df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:04.404518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bb034-2e29-4e65-b74d-a653205a9df5 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:04.404793 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bb034-2e29-4e65-b74d-a653205a9df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:04.405060 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3bb034-2e29-4e65-b74d-a653205a9df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:04.405688 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bb034-2e29-4e65-b74d-a653205a9df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:04.407322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3bb034-2e29-4e65-b74d-a653205a9df5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:04.419312 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1310/2576] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:04.423127 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bb034-2e29-4e65-b74d-a653205a9df5 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:04.434461 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3bb034-2e29-4e65-b74d-a653205a9df5 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:04.444186 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d844e-b856-454b-9e1d-df901e887cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:04.444831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d844e-b856-454b-9e1d-df901e887cec admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:04.446773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d844e-b856-454b-9e1d-df901e887cec admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:04.448366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d844e-b856-454b-9e1d-df901e887cec admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:04.454765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d844e-b856-454b-9e1d-df901e887cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:04.454765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d844e-b856-454b-9e1d-df901e887cec admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:04.465509 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a01b476b-ac27-4d20-965d-3a67a9287f1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a99a537b84ac48c3a7d606e0e2f922b9', 'request_id': 'req-a01b476b-ac27-4d20-965d-3a67a9287f1f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:04.472970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1311/2577] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:04.474103 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a01b476b-ac27-4d20-965d-3a67a9287f1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a99a537b84ac48c3a7d606e0e2f922b9', 'request_id': 'req-a01b476b-ac27-4d20-965d-3a67a9287f1f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:04.478938 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d844e-b856-454b-9e1d-df901e887cec admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:04.481065 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a01b476b-ac27-4d20-965d-3a67a9287f1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a99a537b84ac48c3a7d606e0e2f922b9', 'request_id': 'req-a01b476b-ac27-4d20-965d-3a67a9287f1f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:04.502702 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d844e-b856-454b-9e1d-df901e887cec admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:04.580085 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a01b476b-ac27-4d20-965d-3a67a9287f1f admin admin] The token cache is being invalidate because project a99a537b84ac48c3a7d606e0e2f922b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:04.580437 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a01b476b-ac27-4d20-965d-3a67a9287f1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:04.581219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-a01b476b-ac27-4d20-965d-3a67a9287f1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:04.586796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1267/2578] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:04 2022] DELETE /identity/v3/projects/a99a537b84ac48c3a7d606e0e2f922b9 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:04.592691 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968817f-9557-4aae-ba98-6e2143d0e766 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:04.593160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968817f-9557-4aae-ba98-6e2143d0e766 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:04.593540 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1968817f-9557-4aae-ba98-6e2143d0e766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:04.597489 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1940c5-3c3b-4f7e-8d4d-4593722b7809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:04.598579 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1940c5-3c3b-4f7e-8d4d-4593722b7809 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:04.598579 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1940c5-3c3b-4f7e-8d4d-4593722b7809 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:04.598802 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1940c5-3c3b-4f7e-8d4d-4593722b7809 admin admin] PATH_INFO: `/v3/projects/29de914be2644251b09595f0c20abb7f/users/41e5355a524b4e50885519a71c11530c/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:04.620472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1940c5-3c3b-4f7e-8d4d-4593722b7809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29de914be2644251b09595f0c20abb7f, user_id=41e5355a524b4e50885519a71c11530c, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:04.623197 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1940c5-3c3b-4f7e-8d4d-4593722b7809 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:04.627668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-1968817f-9557-4aae-ba98-6e2143d0e766 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:04.628194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:04.630249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1968817f-9557-4aae-ba98-6e2143d0e766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:04.650302 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1268/2579] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:04.655029 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f4330-ca00-48fd-aae2-085f2d561883 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:04.655425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f4330-ca00-48fd-aae2-085f2d561883 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:04.655788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7f4330-ca00-48fd-aae2-085f2d561883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:04.677834 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-bc7f4330-ca00-48fd-aae2-085f2d561883 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:04.679078 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-bc7f4330-ca00-48fd-aae2-085f2d561883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:04.688544 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1269/2580] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:04.710594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1312/2581] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:04 2022] PUT /identity/v3/projects/29de914be2644251b09595f0c20abb7f/users/41e5355a524b4e50885519a71c11530c/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:04.711210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f4330-ca00-48fd-aae2-085f2d561883 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:04.711935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384df207-bf93-4417-8906-d586c36b4187 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:04.712150 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384df207-bf93-4417-8906-d586c36b4187 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:04.712428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384df207-bf93-4417-8906-d586c36b4187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:04.715204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7f4330-ca00-48fd-aae2-085f2d561883 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:04.734966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-384df207-bf93-4417-8906-d586c36b4187 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:04.737104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-384df207-bf93-4417-8906-d586c36b4187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:04.787136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994517f-9802-494d-a9c5-ecfdf2626ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:04.787759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994517f-9802-494d-a9c5-ecfdf2626ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:04.788041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994517f-9802-494d-a9c5-ecfdf2626ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:04.788320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994517f-9802-494d-a9c5-ecfdf2626ef8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:04.788974 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994517f-9802-494d-a9c5-ecfdf2626ef8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:04:04.789878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994517f-9802-494d-a9c5-ecfdf2626ef8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:04.792924 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994517f-9802-494d-a9c5-ecfdf2626ef8 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:04.811359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1270/2582] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:04:04.813510 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1313/2583] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:04.818232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994517f-9802-494d-a9c5-ecfdf2626ef8 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:04.823738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384df207-bf93-4417-8906-d586c36b4187 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:04.829206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994517f-9802-494d-a9c5-ecfdf2626ef8 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:04.833115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-384df207-bf93-4417-8906-d586c36b4187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:04.834100 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac3ba1-c5d8-4639-8e51-07b88085c4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:04.834705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac3ba1-c5d8-4639-8e51-07b88085c4ad admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:04.835076 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac3ba1-c5d8-4639-8e51-07b88085c4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:04.835345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac3ba1-c5d8-4639-8e51-07b88085c4ad admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:04.836007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac3ba1-c5d8-4639-8e51-07b88085c4ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:04.838441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac3ba1-c5d8-4639-8e51-07b88085c4ad admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:04.856538 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:04.898864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384df207-bf93-4417-8906-d586c36b4187 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:04.902879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94599478-ffb9-4c90-8db2-b99e5f284f63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:04.903574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94599478-ffb9-4c90-8db2-b99e5f284f63 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:04.903910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94599478-ffb9-4c90-8db2-b99e5f284f63 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:04.904257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94599478-ffb9-4c90-8db2-b99e5f284f63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:04.919856 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94599478-ffb9-4c90-8db2-b99e5f284f63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:04.921334 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94599478-ffb9-4c90-8db2-b99e5f284f63 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:04.933191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1314/2584] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:04:04.963193 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1271/2585] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:04 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:04.965878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94599478-ffb9-4c90-8db2-b99e5f284f63 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:04.975743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94599478-ffb9-4c90-8db2-b99e5f284f63 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:05.043169 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee3f55-0dfb-4353-a917-249daccc55ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:05.043866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee3f55-0dfb-4353-a917-249daccc55ae admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:05.044147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee3f55-0dfb-4353-a917-249daccc55ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:05.044376 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ee3f55-0dfb-4353-a917-249daccc55ae admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:05.045065 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee3f55-0dfb-4353-a917-249daccc55ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:05.047282 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ee3f55-0dfb-4353-a917-249daccc55ae admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:05.147574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1315/2586] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:04 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:05.152340 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac3ba1-c5d8-4639-8e51-07b88085c4ad admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:05.165020 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac3ba1-c5d8-4639-8e51-07b88085c4ad admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:05.169921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d789fb-802a-43ea-85f6-fbb618775e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:05.170527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d789fb-802a-43ea-85f6-fbb618775e93 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:05.170805 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d789fb-802a-43ea-85f6-fbb618775e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:05.171097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d789fb-802a-43ea-85f6-fbb618775e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:05.171717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d789fb-802a-43ea-85f6-fbb618775e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:05.173308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d789fb-802a-43ea-85f6-fbb618775e93 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:05.184084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1272/2587] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:05.187370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee3f55-0dfb-4353-a917-249daccc55ae admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:05.197501 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ee3f55-0dfb-4353-a917-249daccc55ae admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:05.202285 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694b53b-c200-46e6-998a-1ce30f71ed39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:05.202851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694b53b-c200-46e6-998a-1ce30f71ed39 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:05.203132 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694b53b-c200-46e6-998a-1ce30f71ed39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:05.203404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694b53b-c200-46e6-998a-1ce30f71ed39 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:05.204045 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694b53b-c200-46e6-998a-1ce30f71ed39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:05.205633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694b53b-c200-46e6-998a-1ce30f71ed39 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:05.216566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1316/2588] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:05.221575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d789fb-802a-43ea-85f6-fbb618775e93 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:05.232290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d789fb-802a-43ea-85f6-fbb618775e93 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:05.237003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffefee46-2f3e-4cf6-a5d7-ec14a3c2b9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:05.237664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefee46-2f3e-4cf6-a5d7-ec14a3c2b9b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:05.237969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefee46-2f3e-4cf6-a5d7-ec14a3c2b9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:05.238246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffefee46-2f3e-4cf6-a5d7-ec14a3c2b9b8 admin admin] PATH_INFO: `/v3/projects/f0f957a134db430aaf6d757af6f59b4e/users/ab217b94278d4c6baea8f28d9c7c77fa/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:05.252641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffefee46-2f3e-4cf6-a5d7-ec14a3c2b9b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f957a134db430aaf6d757af6f59b4e, user_id=ab217b94278d4c6baea8f28d9c7c77fa, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:05.255413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffefee46-2f3e-4cf6-a5d7-ec14a3c2b9b8 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:05.301260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694b53b-c200-46e6-998a-1ce30f71ed39 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:05.303067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d694b53b-c200-46e6-998a-1ce30f71ed39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:05.316386 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694b53b-c200-46e6-998a-1ce30f71ed39 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:05.321052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4309519-8028-402d-9180-f5f7a16578b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:05.321672 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4309519-8028-402d-9180-f5f7a16578b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:05.321984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4309519-8028-402d-9180-f5f7a16578b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:05.322256 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4309519-8028-402d-9180-f5f7a16578b3 admin admin] PATH_INFO: `/v3/users/be0920bd9b934f039e5e4de8039346b8` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:05.323396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4309519-8028-402d-9180-f5f7a16578b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be0920bd9b934f039e5e4de8039346b8)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:05.325297 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4309519-8028-402d-9180-f5f7a16578b3 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:05.339407 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:05.424051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1273/2589] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:05 2022] PUT /identity/v3/projects/f0f957a134db430aaf6d757af6f59b4e/users/ab217b94278d4c6baea8f28d9c7c77fa/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:05.424051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16cb2d-6bde-4bea-963d-a71566244150 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:05.424051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16cb2d-6bde-4bea-963d-a71566244150 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:05.424051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16cb2d-6bde-4bea-963d-a71566244150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:05.450550 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-aa16cb2d-6bde-4bea-963d-a71566244150 None None] MFA Rules not processed for user `ab217b94278d4c6baea8f28d9c7c77fa`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:05.451689 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-aa16cb2d-6bde-4bea-963d-a71566244150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:05.514455 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1274/2590] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:05.526330 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16cb2d-6bde-4bea-963d-a71566244150 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:05.527696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-aa16cb2d-6bde-4bea-963d-a71566244150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:05.585849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16cb2d-6bde-4bea-963d-a71566244150 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:05.590206 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a713e-7e16-4e31-bca6-161efda67d51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:05.590814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a713e-7e16-4e31-bca6-161efda67d51 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:05.591094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a713e-7e16-4e31-bca6-161efda67d51 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:05.591369 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a713e-7e16-4e31-bca6-161efda67d51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:05.600092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a713e-7e16-4e31-bca6-161efda67d51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:05.601570 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a713e-7e16-4e31-bca6-161efda67d51 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:05.620562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b4309519-8028-402d-9180-f5f7a16578b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be0920bd9b934f039e5e4de8039346b8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:05.625969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1275/2591] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:04:05.636180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a713e-7e16-4e31-bca6-161efda67d51 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:05.707935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a713e-7e16-4e31-bca6-161efda67d51 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:05.713948 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e74d5-8b0b-4940-9229-897556304304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:05.714591 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e74d5-8b0b-4940-9229-897556304304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:05.714869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e74d5-8b0b-4940-9229-897556304304 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:05.715141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e74d5-8b0b-4940-9229-897556304304 admin admin] PATH_INFO: `/v3/users/ab217b94278d4c6baea8f28d9c7c77fa` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:05.716531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e74d5-8b0b-4940-9229-897556304304 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab217b94278d4c6baea8f28d9c7c77fa)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:05.718521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0e74d5-8b0b-4940-9229-897556304304 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:05.723904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b4309519-8028-402d-9180-f5f7a16578b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be0920bd9b934f039e5e4de8039346b8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:05.739069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b4309519-8028-402d-9180-f5f7a16578b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be0920bd9b934f039e5e4de8039346b8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:05.740058 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b4309519-8028-402d-9180-f5f7a16578b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be0920bd9b934f039e5e4de8039346b8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:05.740723 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b4309519-8028-402d-9180-f5f7a16578b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be0920bd9b934f039e5e4de8039346b8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:05.753472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1317/2592] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:05 2022] DELETE /identity/v3/users/be0920bd9b934f039e5e4de8039346b8 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:05.979922 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8b0e74d5-8b0b-4940-9229-897556304304 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab217b94278d4c6baea8f28d9c7c77fa'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:05.993410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8b0e74d5-8b0b-4940-9229-897556304304 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab217b94278d4c6baea8f28d9c7c77fa'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:05.999719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4309519-8028-402d-9180-f5f7a16578b3 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:06.001167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b4309519-8028-402d-9180-f5f7a16578b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:06.077699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4309519-8028-402d-9180-f5f7a16578b3 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:06.083129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814020db-0784-4305-88f7-7b4c3ccdcee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:06.083673 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814020db-0784-4305-88f7-7b4c3ccdcee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:06.083935 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814020db-0784-4305-88f7-7b4c3ccdcee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:06.084203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814020db-0784-4305-88f7-7b4c3ccdcee3 admin admin] PATH_INFO: `/v3/projects/5d8e3f408bde455b86274e11fa626eb4` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:06.085354 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814020db-0784-4305-88f7-7b4c3ccdcee3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d8e3f408bde455b86274e11fa626eb4)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:06.087614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814020db-0784-4305-88f7-7b4c3ccdcee3 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:06.089766 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8b0e74d5-8b0b-4940-9229-897556304304 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab217b94278d4c6baea8f28d9c7c77fa'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:06.091009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8b0e74d5-8b0b-4940-9229-897556304304 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab217b94278d4c6baea8f28d9c7c77fa'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:06.099701 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-8b0e74d5-8b0b-4940-9229-897556304304 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab217b94278d4c6baea8f28d9c7c77fa'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:06.106717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1276/2593] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:05 2022] DELETE /identity/v3/users/ab217b94278d4c6baea8f28d9c7c77fa => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:06.110994 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd207c1-78be-4867-9a51-a27009bb4086 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:06.111280 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd207c1-78be-4867-9a51-a27009bb4086 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:06.111557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd207c1-78be-4867-9a51-a27009bb4086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:06.132068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-8fd207c1-78be-4867-9a51-a27009bb4086 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:06.133951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8fd207c1-78be-4867-9a51-a27009bb4086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:06.192792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-814020db-0784-4305-88f7-7b4c3ccdcee3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d8e3f408bde455b86274e11fa626eb4', 'request_id': 'req-814020db-0784-4305-88f7-7b4c3ccdcee3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:06.194915 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1277/2594] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:06.203824 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:06.205705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd207c1-78be-4867-9a51-a27009bb4086 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:06.207096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8fd207c1-78be-4867-9a51-a27009bb4086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:06.260642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-814020db-0784-4305-88f7-7b4c3ccdcee3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d8e3f408bde455b86274e11fa626eb4', 'request_id': 'req-814020db-0784-4305-88f7-7b4c3ccdcee3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:06.268103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-814020db-0784-4305-88f7-7b4c3ccdcee3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d8e3f408bde455b86274e11fa626eb4', 'request_id': 'req-814020db-0784-4305-88f7-7b4c3ccdcee3'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:06.271871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd207c1-78be-4867-9a51-a27009bb4086 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:06.280165 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-814020db-0784-4305-88f7-7b4c3ccdcee3 admin admin] The token cache is being invalidate because project 5d8e3f408bde455b86274e11fa626eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:06.280455 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-814020db-0784-4305-88f7-7b4c3ccdcee3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:06.281106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-814020db-0784-4305-88f7-7b4c3ccdcee3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:06.286223 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1318/2595] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:05 2022] DELETE /identity/v3/projects/5d8e3f408bde455b86274e11fa626eb4 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:06.287497 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885a51a-f5d4-4401-8bc6-c8927e2bd348 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:06.287866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885a51a-f5d4-4401-8bc6-c8927e2bd348 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:06.288090 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885a51a-f5d4-4401-8bc6-c8927e2bd348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:06.308902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-f885a51a-f5d4-4401-8bc6-c8927e2bd348 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:06.310132 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f885a51a-f5d4-4401-8bc6-c8927e2bd348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:06.327390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5ebfe-1da3-4dc2-9026-457638b4217e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:06.328096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5ebfe-1da3-4dc2-9026-457638b4217e service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:06.328355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5ebfe-1da3-4dc2-9026-457638b4217e service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:06.328621 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5ebfe-1da3-4dc2-9026-457638b4217e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:06.330542 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-aff5ebfe-1da3-4dc2-9026-457638b4217e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:06.354284 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:06.374667 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1319/2596] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:06.383615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53ae8b-72d9-4280-a450-96106725490f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:06.383899 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53ae8b-72d9-4280-a450-96106725490f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:06.384173 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53ae8b-72d9-4280-a450-96106725490f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:06.394597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5ebfe-1da3-4dc2-9026-457638b4217e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:06.396037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5ebfe-1da3-4dc2-9026-457638b4217e service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:06.402977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-fb53ae8b-72d9-4280-a450-96106725490f None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:06.405433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fb53ae8b-72d9-4280-a450-96106725490f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:06.405932 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1278/2597] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:04:06.411950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1320/2598] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:06.416677 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5ebfe-1da3-4dc2-9026-457638b4217e service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:06.426095 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5ebfe-1da3-4dc2-9026-457638b4217e service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:06.431377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a3edf-00bc-47e5-9336-b6621b4e035f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:06.431971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a3edf-00bc-47e5-9336-b6621b4e035f admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:06.432247 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a3edf-00bc-47e5-9336-b6621b4e035f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:06.432519 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a3edf-00bc-47e5-9336-b6621b4e035f admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:06.433151 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a3edf-00bc-47e5-9336-b6621b4e035f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:04:06.433452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a3edf-00bc-47e5-9336-b6621b4e035f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:06.435090 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a3edf-00bc-47e5-9336-b6621b4e035f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:06.447470 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1279/2599] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 20:04:06.451898 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53ae8b-72d9-4280-a450-96106725490f None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:06.462150 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53ae8b-72d9-4280-a450-96106725490f None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:06.466875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5a8e78-11d7-4372-9c81-015ba6853cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:06.467473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a8e78-11d7-4372-9c81-015ba6853cf4 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:06.467753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a8e78-11d7-4372-9c81-015ba6853cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:06.468024 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5a8e78-11d7-4372-9c81-015ba6853cf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:06.468656 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a8e78-11d7-4372-9c81-015ba6853cf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:06.470659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5a8e78-11d7-4372-9c81-015ba6853cf4 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:06.595271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1321/2600] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:06 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:06.599344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a3edf-00bc-47e5-9336-b6621b4e035f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:06.609962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a3edf-00bc-47e5-9336-b6621b4e035f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:06.634361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d9653-6fb7-4b56-90ca-0b853a936d5a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:06.635207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d9653-6fb7-4b56-90ca-0b853a936d5a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:06.635708 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d9653-6fb7-4b56-90ca-0b853a936d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:06.667718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-333d9653-6fb7-4b56-90ca-0b853a936d5a None None] MFA Rules not processed for user `41e5355a524b4e50885519a71c11530c`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:06.669097 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-333d9653-6fb7-4b56-90ca-0b853a936d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:06.725904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8041dc-ce25-4e3c-a3e6-27181a826045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:06.733989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8041dc-ce25-4e3c-a3e6-27181a826045 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:06.733989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8041dc-ce25-4e3c-a3e6-27181a826045 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:06.733989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8041dc-ce25-4e3c-a3e6-27181a826045 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:06.733989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8041dc-ce25-4e3c-a3e6-27181a826045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:06.741236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8041dc-ce25-4e3c-a3e6-27181a826045 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:06.747881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1322/2601] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:06.752658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d9653-6fb7-4b56-90ca-0b853a936d5a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:06.754462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-333d9653-6fb7-4b56-90ca-0b853a936d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:06.774344 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d9653-6fb7-4b56-90ca-0b853a936d5a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:06.779692 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5613025-9d9b-48d1-a299-126fa1b1cab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:06.780585 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5613025-9d9b-48d1-a299-126fa1b1cab2 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:06.781190 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5613025-9d9b-48d1-a299-126fa1b1cab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:06.781343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:06.781884 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5613025-9d9b-48d1-a299-126fa1b1cab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:06.782653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5613025-9d9b-48d1-a299-126fa1b1cab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:06.784619 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5613025-9d9b-48d1-a299-126fa1b1cab2 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:06.830891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1280/2602] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:06 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:06.831715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8041dc-ce25-4e3c-a3e6-27181a826045 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:06.842135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8041dc-ce25-4e3c-a3e6-27181a826045 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:06.846713 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3792159-3baa-4f9c-973e-3d56bdd29779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:06.847595 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3792159-3baa-4f9c-973e-3d56bdd29779 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:06.847595 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3792159-3baa-4f9c-973e-3d56bdd29779 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:06.847828 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3792159-3baa-4f9c-973e-3d56bdd29779 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:06.848431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3792159-3baa-4f9c-973e-3d56bdd29779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:06.850042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3792159-3baa-4f9c-973e-3d56bdd29779 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:06.861005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1281/2603] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:06.865018 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3792159-3baa-4f9c-973e-3d56bdd29779 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:06.876465 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3792159-3baa-4f9c-973e-3d56bdd29779 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:06.880849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7419957-bd64-4414-a8b3-deb1e42e2f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:06.881553 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419957-bd64-4414-a8b3-deb1e42e2f05 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:06.882093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419957-bd64-4414-a8b3-deb1e42e2f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:06.882334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7419957-bd64-4414-a8b3-deb1e42e2f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:06.882913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7419957-bd64-4414-a8b3-deb1e42e2f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:06.884583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7419957-bd64-4414-a8b3-deb1e42e2f05 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:06.885055 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1323/2604] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:06 2022] POST /identity/v3/projects => generated 371 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:06.888804 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5613025-9d9b-48d1-a299-126fa1b1cab2 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:06.897179 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1282/2605] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:06.901586 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7419957-bd64-4414-a8b3-deb1e42e2f05 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:06.912761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7419957-bd64-4414-a8b3-deb1e42e2f05 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:06.963597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5613025-9d9b-48d1-a299-126fa1b1cab2 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:06.968235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4aa024-e779-4912-aab1-d745056f556c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:06.969008 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4aa024-e779-4912-aab1-d745056f556c admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:06.969305 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4aa024-e779-4912-aab1-d745056f556c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:06.969578 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4aa024-e779-4912-aab1-d745056f556c admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:06.970288 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4aa024-e779-4912-aab1-d745056f556c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:06.972181 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4aa024-e779-4912-aab1-d745056f556c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:06.974588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c39d9-c981-424f-89c7-6461bed92eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:06.975246 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c39d9-c981-424f-89c7-6461bed92eca admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:06.975521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c39d9-c981-424f-89c7-6461bed92eca admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:06.975791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243c39d9-c981-424f-89c7-6461bed92eca admin admin] PATH_INFO: `/v3/projects/54de5ffcf73f4cf69a05c09447e5307e/users/b8ca6d7488e9455697d4319bd4fa227f/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:06.991094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c39d9-c981-424f-89c7-6461bed92eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=54de5ffcf73f4cf69a05c09447e5307e, user_id=b8ca6d7488e9455697d4319bd4fa227f, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:06.994007 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243c39d9-c981-424f-89c7-6461bed92eca admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:07.053506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1324/2606] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:06 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:07.054581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4aa024-e779-4912-aab1-d745056f556c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:07.056032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6d4aa024-e779-4912-aab1-d745056f556c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:07.068086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4aa024-e779-4912-aab1-d745056f556c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:07.073230 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:07.073864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:07.074141 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:07.074407 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] PATH_INFO: `/v3/projects/f0f957a134db430aaf6d757af6f59b4e` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:07.075495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0f957a134db430aaf6d757af6f59b4e)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:07.077374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:07.084323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1283/2607] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:06 2022] PUT /identity/v3/projects/54de5ffcf73f4cf69a05c09447e5307e/users/b8ca6d7488e9455697d4319bd4fa227f/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:07.085318 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c39d9-c981-424f-89c7-6461bed92eca admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:07.147493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243c39d9-c981-424f-89c7-6461bed92eca admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:07.153320 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb26486-fd39-43c7-8fb3-d5f1509e34e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:07.153973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb26486-fd39-43c7-8fb3-d5f1509e34e4 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:07.154255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb26486-fd39-43c7-8fb3-d5f1509e34e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:07.154528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb26486-fd39-43c7-8fb3-d5f1509e34e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:07.155162 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb26486-fd39-43c7-8fb3-d5f1509e34e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:07.156925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb26486-fd39-43c7-8fb3-d5f1509e34e4 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:07.166613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1284/2608] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:07.167928 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad9cbc-6d5f-4df4-9bd4-ff2b90a55b60 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:07.168212 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad9cbc-6d5f-4df4-9bd4-ff2b90a55b60 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:07.168487 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad9cbc-6d5f-4df4-9bd4-ff2b90a55b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:07.195045 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0f957a134db430aaf6d757af6f59b4e', 'request_id': 'req-ac07239e-65ff-429c-be16-3e056099667c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:07.196048 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-00ad9cbc-6d5f-4df4-9bd4-ff2b90a55b60 None None] MFA Rules not processed for user `b8ca6d7488e9455697d4319bd4fa227f`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:07.197356 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-00ad9cbc-6d5f-4df4-9bd4-ff2b90a55b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:07.260133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1285/2609] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:07.260938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad9cbc-6d5f-4df4-9bd4-ff2b90a55b60 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:07.272220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0f957a134db430aaf6d757af6f59b4e', 'request_id': 'req-ac07239e-65ff-429c-be16-3e056099667c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:07.280328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0f957a134db430aaf6d757af6f59b4e', 'request_id': 'req-ac07239e-65ff-429c-be16-3e056099667c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:07.293258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] The token cache is being invalidate because project f0f957a134db430aaf6d757af6f59b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:07.293258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:07.293258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:07.293258 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1325/2610] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:07 2022] DELETE /identity/v3/projects/f0f957a134db430aaf6d757af6f59b4e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:07.293833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:07.295103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:07.322675 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad9cbc-6d5f-4df4-9bd4-ff2b90a55b60 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:07.327228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e07ad-35fb-4a5f-b061-c5d086d23bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:07.327884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e07ad-35fb-4a5f-b061-c5d086d23bbc admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:07.328159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e07ad-35fb-4a5f-b061-c5d086d23bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:07.328451 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e07ad-35fb-4a5f-b061-c5d086d23bbc admin admin] PATH_INFO: `/v3/projects/634e71816d054ebaa641b2f5a81fdad1/users/bd8613650a1147f3ac9d008b6591cdf8/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:07.343106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e07ad-35fb-4a5f-b061-c5d086d23bbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=634e71816d054ebaa641b2f5a81fdad1, user_id=bd8613650a1147f3ac9d008b6591cdf8, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:07.345817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e07ad-35fb-4a5f-b061-c5d086d23bbc admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:07.358618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07239e-65ff-429c-be16-3e056099667c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:07.363233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec6980-cfe0-44ee-8bbc-9512d76d7624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:07.363832 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec6980-cfe0-44ee-8bbc-9512d76d7624 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:07.364106 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec6980-cfe0-44ee-8bbc-9512d76d7624 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:07.364370 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fec6980-cfe0-44ee-8bbc-9512d76d7624 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:07.364980 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec6980-cfe0-44ee-8bbc-9512d76d7624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:07.366876 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fec6980-cfe0-44ee-8bbc-9512d76d7624 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:07.372527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:07.472207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1286/2611] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:07 2022] PUT /identity/v3/projects/634e71816d054ebaa641b2f5a81fdad1/users/bd8613650a1147f3ac9d008b6591cdf8/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:07.476294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f21b6-8a18-4be8-a306-173a2da25852 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:07.476580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f21b6-8a18-4be8-a306-173a2da25852 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:07.476892 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f21b6-8a18-4be8-a306-173a2da25852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:07.493970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1326/2612] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:07 2022] POST /identity/v3/projects => generated 387 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:07.493970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec6980-cfe0-44ee-8bbc-9512d76d7624 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:07.495665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1fec6980-cfe0-44ee-8bbc-9512d76d7624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:07.498730 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-ca2f21b6-8a18-4be8-a306-173a2da25852 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:07.500029 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ca2f21b6-8a18-4be8-a306-173a2da25852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:07.559280 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1287/2613] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:07.559592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fec6980-cfe0-44ee-8bbc-9512d76d7624 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:07.560349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f21b6-8a18-4be8-a306-173a2da25852 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:07.569764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60843dc8-7bd1-4465-ada2-a968e02e80e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:07.570407 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60843dc8-7bd1-4465-ada2-a968e02e80e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:07.571250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60843dc8-7bd1-4465-ada2-a968e02e80e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:07.571526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60843dc8-7bd1-4465-ada2-a968e02e80e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:07.572170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60843dc8-7bd1-4465-ada2-a968e02e80e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:07.573916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60843dc8-7bd1-4465-ada2-a968e02e80e9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:07.575441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f21b6-8a18-4be8-a306-173a2da25852 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:07.580191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fff5e-b917-46aa-b2cd-1a128c76a28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:07.580788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fff5e-b917-46aa-b2cd-1a128c76a28a admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:07.581062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fff5e-b917-46aa-b2cd-1a128c76a28a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:07.581333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fff5e-b917-46aa-b2cd-1a128c76a28a admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:07.581969 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fff5e-b917-46aa-b2cd-1a128c76a28a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:07.583848 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fff5e-b917-46aa-b2cd-1a128c76a28a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:07.584076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1327/2614] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:07.585387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5622f28-9c17-44a3-a92c-20dcbef0db16 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:07.585681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5622f28-9c17-44a3-a92c-20dcbef0db16 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:07.586030 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5622f28-9c17-44a3-a92c-20dcbef0db16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:07.605785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c5622f28-9c17-44a3-a92c-20dcbef0db16 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:07.607006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c5622f28-9c17-44a3-a92c-20dcbef0db16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:07.616131 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1328/2615] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:07.616866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5622f28-9c17-44a3-a92c-20dcbef0db16 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:07.626523 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5622f28-9c17-44a3-a92c-20dcbef0db16 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:07.631216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843db21-4c04-4d18-821a-5729409f4b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:07.631831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843db21-4c04-4d18-821a-5729409f4b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:07.632104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843db21-4c04-4d18-821a-5729409f4b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:07.632372 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843db21-4c04-4d18-821a-5729409f4b8e admin admin] PATH_INFO: `/v3/projects/634e71816d054ebaa641b2f5a81fdad1/users/bd8613650a1147f3ac9d008b6591cdf8/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:07.635055 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843db21-4c04-4d18-821a-5729409f4b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=634e71816d054ebaa641b2f5a81fdad1, user_id=bd8613650a1147f3ac9d008b6591cdf8, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:07.637948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843db21-4c04-4d18-821a-5729409f4b8e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:07.647794 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1329/2616] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:07 2022] PUT /identity/v3/projects/634e71816d054ebaa641b2f5a81fdad1/users/bd8613650a1147f3ac9d008b6591cdf8/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:07.648602 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843db21-4c04-4d18-821a-5729409f4b8e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:07.677276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1288/2617] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:07 2022] POST /identity/v3/users => generated 300 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:07.678479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fff5e-b917-46aa-b2cd-1a128c76a28a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:07.688733 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fff5e-b917-46aa-b2cd-1a128c76a28a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:07.712682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843db21-4c04-4d18-821a-5729409f4b8e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:07.717685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d5135-20b7-4d95-b147-c546a381f5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:07.718308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d5135-20b7-4d95-b147-c546a381f5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:07.718583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d5135-20b7-4d95-b147-c546a381f5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:07.718852 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619d5135-20b7-4d95-b147-c546a381f5d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:07.719477 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d5135-20b7-4d95-b147-c546a381f5d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:04:07.719837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d5135-20b7-4d95-b147-c546a381f5d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:07.721410 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619d5135-20b7-4d95-b147-c546a381f5d1 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:07.734820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1330/2618] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:04:07.739114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d5135-20b7-4d95-b147-c546a381f5d1 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:07.749235 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3ee3c-829d-45c4-83b9-e17ee4d8ef36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:07.749721 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619d5135-20b7-4d95-b147-c546a381f5d1 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:07.749894 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3ee3c-829d-45c4-83b9-e17ee4d8ef36 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:07.750139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3ee3c-829d-45c4-83b9-e17ee4d8ef36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:07.750410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3ee3c-829d-45c4-83b9-e17ee4d8ef36 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:07.751136 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3ee3c-829d-45c4-83b9-e17ee4d8ef36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:07.753737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3ee3c-829d-45c4-83b9-e17ee4d8ef36 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:07.759793 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:07.766849 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fdc94-c2f8-4def-a81a-536d31662979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:07.767777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fdc94-c2f8-4def-a81a-536d31662979 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:07.767777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fdc94-c2f8-4def-a81a-536d31662979 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:07.767991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8fdc94-c2f8-4def-a81a-536d31662979 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:07.768618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8fdc94-c2f8-4def-a81a-536d31662979 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:07.770565 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8fdc94-c2f8-4def-a81a-536d31662979 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:07.773178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1289/2619] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:07.777467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3ee3c-829d-45c4-83b9-e17ee4d8ef36 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:07.788474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3ee3c-829d-45c4-83b9-e17ee4d8ef36 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:07.794179 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75478c1e-f994-47a2-a6ac-634374ac33fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:07.799807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75478c1e-f994-47a2-a6ac-634374ac33fb admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:07.799807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75478c1e-f994-47a2-a6ac-634374ac33fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:07.799807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75478c1e-f994-47a2-a6ac-634374ac33fb admin admin] PATH_INFO: `/v3/projects/dc13fe1888574bfb81e84328c1bb211d/users/b551ea1a13014d2ab27e3b26b3d45474/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:07.811193 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75478c1e-f994-47a2-a6ac-634374ac33fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc13fe1888574bfb81e84328c1bb211d, user_id=b551ea1a13014d2ab27e3b26b3d45474, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:07.813883 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75478c1e-f994-47a2-a6ac-634374ac33fb admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:07.890906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1331/2620] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:07 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:07.895171 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fdc94-c2f8-4def-a81a-536d31662979 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:07.919425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1290/2621] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:07 2022] PUT /identity/v3/projects/dc13fe1888574bfb81e84328c1bb211d/users/b551ea1a13014d2ab27e3b26b3d45474/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:07.928998 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75478c1e-f994-47a2-a6ac-634374ac33fb admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:07.974313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8fdc94-c2f8-4def-a81a-536d31662979 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:08.017563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:08.019729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75478c1e-f994-47a2-a6ac-634374ac33fb admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:08.024811 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7b9a9-d7fd-4441-9e0d-f9416bcb241d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:08.025418 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7b9a9-d7fd-4441-9e0d-f9416bcb241d admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:08.025703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7b9a9-d7fd-4441-9e0d-f9416bcb241d admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:08.026229 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7b9a9-d7fd-4441-9e0d-f9416bcb241d admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:08.026634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7b9a9-d7fd-4441-9e0d-f9416bcb241d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:08.028228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7b9a9-d7fd-4441-9e0d-f9416bcb241d admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:08.038547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e7fae-8c83-4b9f-97ea-164d025526d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:08.038789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1291/2622] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:08.040273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e7fae-8c83-4b9f-97ea-164d025526d5 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:08.040548 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e7fae-8c83-4b9f-97ea-164d025526d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:08.040815 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e7fae-8c83-4b9f-97ea-164d025526d5 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:08.041436 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e7fae-8c83-4b9f-97ea-164d025526d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:08.043566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e7fae-8c83-4b9f-97ea-164d025526d5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:08.043807 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7b9a9-d7fd-4441-9e0d-f9416bcb241d admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:08.055734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7b9a9-d7fd-4441-9e0d-f9416bcb241d admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:08.060641 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65123fc2-9d5f-481d-941d-fc2e972e7ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:08.061352 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65123fc2-9d5f-481d-941d-fc2e972e7ae0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:08.068300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65123fc2-9d5f-481d-941d-fc2e972e7ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:08.068300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65123fc2-9d5f-481d-941d-fc2e972e7ae0 admin admin] PATH_INFO: `/v3/projects/dc13fe1888574bfb81e84328c1bb211d/users/b551ea1a13014d2ab27e3b26b3d45474/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:08.068300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65123fc2-9d5f-481d-941d-fc2e972e7ae0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc13fe1888574bfb81e84328c1bb211d, user_id=b551ea1a13014d2ab27e3b26b3d45474, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:08.068300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65123fc2-9d5f-481d-941d-fc2e972e7ae0 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:08.078563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1292/2623] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:08 2022] PUT /identity/v3/projects/dc13fe1888574bfb81e84328c1bb211d/users/b551ea1a13014d2ab27e3b26b3d45474/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:08.090110 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2f6cb-0e1e-4fed-a0e0-1b9508ebe92e None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:08.090460 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2f6cb-0e1e-4fed-a0e0-1b9508ebe92e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:08.090740 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb2f6cb-0e1e-4fed-a0e0-1b9508ebe92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:08.119079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-6cb2f6cb-0e1e-4fed-a0e0-1b9508ebe92e None None] MFA Rules not processed for user `b551ea1a13014d2ab27e3b26b3d45474`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:08.120288 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6cb2f6cb-0e1e-4fed-a0e0-1b9508ebe92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:08.125084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1332/2624] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:07 2022] POST /identity/v3/users => generated 289 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:08.128485 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e7fae-8c83-4b9f-97ea-164d025526d5 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:08.139928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e7fae-8c83-4b9f-97ea-164d025526d5 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:08.197680 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:08.203314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43158895-bbe1-4c93-a96a-cd8d0b38fc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:08.203939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43158895-bbe1-4c93-a96a-cd8d0b38fc9a admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:08.204219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43158895-bbe1-4c93-a96a-cd8d0b38fc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:08.204491 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43158895-bbe1-4c93-a96a-cd8d0b38fc9a admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:08.205307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43158895-bbe1-4c93-a96a-cd8d0b38fc9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:08.206982 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43158895-bbe1-4c93-a96a-cd8d0b38fc9a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:08.210195 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1293/2625] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:08 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:08.221406 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1333/2626] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:08.222664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2f6cb-0e1e-4fed-a0e0-1b9508ebe92e None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:08.225420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43158895-bbe1-4c93-a96a-cd8d0b38fc9a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:08.226550 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6cb2f6cb-0e1e-4fed-a0e0-1b9508ebe92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:08.238066 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43158895-bbe1-4c93-a96a-cd8d0b38fc9a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:08.243835 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451ba6e-9c1a-4600-be95-b2fc01d6cc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:08.244534 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451ba6e-9c1a-4600-be95-b2fc01d6cc42 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:08.244879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451ba6e-9c1a-4600-be95-b2fc01d6cc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:08.245221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2451ba6e-9c1a-4600-be95-b2fc01d6cc42 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:08.245984 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451ba6e-9c1a-4600-be95-b2fc01d6cc42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:08.247846 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2451ba6e-9c1a-4600-be95-b2fc01d6cc42 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:08.262420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1334/2627] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:08.266666 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451ba6e-9c1a-4600-be95-b2fc01d6cc42 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:08.277016 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2451ba6e-9c1a-4600-be95-b2fc01d6cc42 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:08.281972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241baa2d-f3fb-4165-8f7e-8a3c09a19753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:08.282638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241baa2d-f3fb-4165-8f7e-8a3c09a19753 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:08.282917 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241baa2d-f3fb-4165-8f7e-8a3c09a19753 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:08.283188 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241baa2d-f3fb-4165-8f7e-8a3c09a19753 admin admin] PATH_INFO: `/v3/projects/ddd31ffab3ea4564aa7b27b1b3ccf95d/users/2283370cbffb4188ab6e5f60243e5a37/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:08.325264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb2f6cb-0e1e-4fed-a0e0-1b9508ebe92e None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:08.329268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf6618-f723-4b42-a405-f34a1f481c08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:08.329925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf6618-f723-4b42-a405-f34a1f481c08 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:08.330204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf6618-f723-4b42-a405-f34a1f481c08 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:08.330478 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbf6618-f723-4b42-a405-f34a1f481c08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:08.341740 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf6618-f723-4b42-a405-f34a1f481c08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:08.343561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbf6618-f723-4b42-a405-f34a1f481c08 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:08.355904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1294/2628] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:08.423546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241baa2d-f3fb-4165-8f7e-8a3c09a19753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddd31ffab3ea4564aa7b27b1b3ccf95d, user_id=2283370cbffb4188ab6e5f60243e5a37, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:08.431118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241baa2d-f3fb-4165-8f7e-8a3c09a19753 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:08.434823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf6618-f723-4b42-a405-f34a1f481c08 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:08.445213 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbf6618-f723-4b42-a405-f34a1f481c08 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:08.450791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff186b1-40aa-4097-b6ac-da0cc31a0fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:08.451392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff186b1-40aa-4097-b6ac-da0cc31a0fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:08.451667 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff186b1-40aa-4097-b6ac-da0cc31a0fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:08.451936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff186b1-40aa-4097-b6ac-da0cc31a0fe4 admin admin] PATH_INFO: `/v3/users/b8ca6d7488e9455697d4319bd4fa227f` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:08.453068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff186b1-40aa-4097-b6ac-da0cc31a0fe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8ca6d7488e9455697d4319bd4fa227f)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:08.455164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff186b1-40aa-4097-b6ac-da0cc31a0fe4 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:08.459736 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:08.575352 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1335/2629] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:08 2022] PUT /identity/v3/projects/ddd31ffab3ea4564aa7b27b1b3ccf95d/users/2283370cbffb4188ab6e5f60243e5a37/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:08.585651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790edaa-4777-4d7e-9ce7-67cd3f35e464 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:08.585966 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790edaa-4777-4d7e-9ce7-67cd3f35e464 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:08.586270 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790edaa-4777-4d7e-9ce7-67cd3f35e464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:08.614879 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1790edaa-4777-4d7e-9ce7-67cd3f35e464 None None] MFA Rules not processed for user `2283370cbffb4188ab6e5f60243e5a37`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:08.615064 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1790edaa-4777-4d7e-9ce7-67cd3f35e464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:08.666319 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3ff186b1-40aa-4097-b6ac-da0cc31a0fe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8ca6d7488e9455697d4319bd4fa227f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:08.679960 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3ff186b1-40aa-4097-b6ac-da0cc31a0fe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8ca6d7488e9455697d4319bd4fa227f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:08.689225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1336/2630] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:08.693617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790edaa-4777-4d7e-9ce7-67cd3f35e464 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:08.706586 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:08.746334 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3ff186b1-40aa-4097-b6ac-da0cc31a0fe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8ca6d7488e9455697d4319bd4fa227f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:08.746952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3ff186b1-40aa-4097-b6ac-da0cc31a0fe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8ca6d7488e9455697d4319bd4fa227f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:08.754278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3ff186b1-40aa-4097-b6ac-da0cc31a0fe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8ca6d7488e9455697d4319bd4fa227f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:08.755915 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790edaa-4777-4d7e-9ce7-67cd3f35e464 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:08.760274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1295/2631] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:08 2022] DELETE /identity/v3/users/b8ca6d7488e9455697d4319bd4fa227f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:08.763356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a47fa-93c6-4af3-9f64-6295b19f6d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:08.764583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3faa1-dd18-4c48-b791-fcadfb73ff5e None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:08.764871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3faa1-dd18-4c48-b791-fcadfb73ff5e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:08.765003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a47fa-93c6-4af3-9f64-6295b19f6d4d admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:08.765146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade3faa1-dd18-4c48-b791-fcadfb73ff5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:08.765419 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a47fa-93c6-4af3-9f64-6295b19f6d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:08.765893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a47fa-93c6-4af3-9f64-6295b19f6d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:08.766728 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a47fa-93c6-4af3-9f64-6295b19f6d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:08.770276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a47fa-93c6-4af3-9f64-6295b19f6d4d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:08.784651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-ade3faa1-dd18-4c48-b791-fcadfb73ff5e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:08.785881 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ade3faa1-dd18-4c48-b791-fcadfb73ff5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:08.850750 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1296/2632] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:08.857991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1337/2633] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:08 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:08.861533 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3faa1-dd18-4c48-b791-fcadfb73ff5e None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:08.861687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a47fa-93c6-4af3-9f64-6295b19f6d4d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:08.863232 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ade3faa1-dd18-4c48-b791-fcadfb73ff5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:08.864473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-047a47fa-93c6-4af3-9f64-6295b19f6d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:08.937175 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade3faa1-dd18-4c48-b791-fcadfb73ff5e None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:08.941247 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a47fa-93c6-4af3-9f64-6295b19f6d4d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:08.947615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee2cb9-1611-45bb-be29-54f2c6de5399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:08.949836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee2cb9-1611-45bb-be29-54f2c6de5399 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:08.949836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee2cb9-1611-45bb-be29-54f2c6de5399 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:08.949836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee2cb9-1611-45bb-be29-54f2c6de5399 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:08.949836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee2cb9-1611-45bb-be29-54f2c6de5399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:08.951869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ee2cb9-1611-45bb-be29-54f2c6de5399 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:08.990661 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c62ac-2ce7-4688-b689-308ec8ac90c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:08.991356 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c62ac-2ce7-4688-b689-308ec8ac90c0 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:08.991703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c62ac-2ce7-4688-b689-308ec8ac90c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:08.992063 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c62ac-2ce7-4688-b689-308ec8ac90c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:09.001485 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c62ac-2ce7-4688-b689-308ec8ac90c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:09.002953 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c62ac-2ce7-4688-b689-308ec8ac90c0 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:09.012515 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1297/2634] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:04:09.013514 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d17356-9aed-4b1e-984a-8790fc101c45 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:09.013777 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d17356-9aed-4b1e-984a-8790fc101c45 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:09.014081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d17356-9aed-4b1e-984a-8790fc101c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:09.038574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1338/2635] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:08 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:09.043311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee2cb9-1611-45bb-be29-54f2c6de5399 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:09.046362 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:09.048536 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-80d17356-9aed-4b1e-984a-8790fc101c45 None None] MFA Rules not processed for user `bd8613650a1147f3ac9d008b6591cdf8`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:09.049619 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-80d17356-9aed-4b1e-984a-8790fc101c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:09.064443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee2cb9-1611-45bb-be29-54f2c6de5399 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:09.070224 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcd9f8-1d9a-48f7-9caf-fb1c035417ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:09.071815 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcd9f8-1d9a-48f7-9caf-fb1c035417ef admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:09.072120 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcd9f8-1d9a-48f7-9caf-fb1c035417ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:09.072529 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcd9f8-1d9a-48f7-9caf-fb1c035417ef admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:09.073177 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcd9f8-1d9a-48f7-9caf-fb1c035417ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:09.075072 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcd9f8-1d9a-48f7-9caf-fb1c035417ef admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:09.088444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1339/2636] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:09.093394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcd9f8-1d9a-48f7-9caf-fb1c035417ef admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:09.105292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcd9f8-1d9a-48f7-9caf-fb1c035417ef admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:09.110397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb210e05-0e23-4434-bc4e-499f7d1298fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:09.111135 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb210e05-0e23-4434-bc4e-499f7d1298fe admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:09.111409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb210e05-0e23-4434-bc4e-499f7d1298fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:09.111685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb210e05-0e23-4434-bc4e-499f7d1298fe admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:09.112397 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb210e05-0e23-4434-bc4e-499f7d1298fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:09.114191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb210e05-0e23-4434-bc4e-499f7d1298fe admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:09.125058 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1340/2637] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:09.129063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb210e05-0e23-4434-bc4e-499f7d1298fe admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:09.134133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1298/2638] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:09.139164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb210e05-0e23-4434-bc4e-499f7d1298fe admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:09.143920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090249cb-9792-4779-a039-fe782a81e94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:09.144583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090249cb-9792-4779-a039-fe782a81e94c admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:09.144830 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090249cb-9792-4779-a039-fe782a81e94c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:09.145098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d17356-9aed-4b1e-984a-8790fc101c45 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:09.145212 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090249cb-9792-4779-a039-fe782a81e94c admin admin] PATH_INFO: `/v3/projects/64d2388928f5436eb2c35be00a28e06e/users/cdf79a7aee3a42c18c90796e5f0c0c60/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:09.146518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-80d17356-9aed-4b1e-984a-8790fc101c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:09.162912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090249cb-9792-4779-a039-fe782a81e94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=64d2388928f5436eb2c35be00a28e06e, user_id=cdf79a7aee3a42c18c90796e5f0c0c60, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:09.166399 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090249cb-9792-4779-a039-fe782a81e94c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:09.206576 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d17356-9aed-4b1e-984a-8790fc101c45 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:09.210225 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a81751-786f-494b-a638-4d28e9e23498 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:09.210831 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a81751-786f-494b-a638-4d28e9e23498 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:09.211910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a81751-786f-494b-a638-4d28e9e23498 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:09.212165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a81751-786f-494b-a638-4d28e9e23498 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:09.214686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-88a81751-786f-494b-a638-4d28e9e23498 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:09.280490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a81751-786f-494b-a638-4d28e9e23498 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:09.282010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a81751-786f-494b-a638-4d28e9e23498 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:09.291693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1299/2639] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:04:09.353181 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:09.369827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1341/2640] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:09 2022] PUT /identity/v3/projects/64d2388928f5436eb2c35be00a28e06e/users/cdf79a7aee3a42c18c90796e5f0c0c60/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:09.443518 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68773a6-7d7e-4d61-9aca-b35814f92956 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:09.443952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68773a6-7d7e-4d61-9aca-b35814f92956 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:09.444331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68773a6-7d7e-4d61-9aca-b35814f92956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:09.474313 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-b68773a6-7d7e-4d61-9aca-b35814f92956 None None] MFA Rules not processed for user `cdf79a7aee3a42c18c90796e5f0c0c60`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:09.475533 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b68773a6-7d7e-4d61-9aca-b35814f92956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:09.539747 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1300/2641] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:09.544435 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090249cb-9792-4779-a039-fe782a81e94c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:09.607116 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090249cb-9792-4779-a039-fe782a81e94c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:09.612058 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549be136-27c4-4fed-b73f-44f2fd6d8722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:09.612659 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be136-27c4-4fed-b73f-44f2fd6d8722 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:09.612929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be136-27c4-4fed-b73f-44f2fd6d8722 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:09.613239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549be136-27c4-4fed-b73f-44f2fd6d8722 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:09.613888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549be136-27c4-4fed-b73f-44f2fd6d8722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:09.615810 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549be136-27c4-4fed-b73f-44f2fd6d8722 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:09.638692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68773a6-7d7e-4d61-9aca-b35814f92956 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:09.638692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b68773a6-7d7e-4d61-9aca-b35814f92956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:09.655549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68773a6-7d7e-4d61-9aca-b35814f92956 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:09.661692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:09.662391 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:09.662692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:09.662965 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] PATH_INFO: `/v3/projects/54de5ffcf73f4cf69a05c09447e5307e` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:09.664163 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54de5ffcf73f4cf69a05c09447e5307e)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:09.666104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:09.690677 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1342/2642] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:09 2022] POST /identity/v3/projects => generated 377 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:09.694746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549be136-27c4-4fed-b73f-44f2fd6d8722 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:09.757392 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549be136-27c4-4fed-b73f-44f2fd6d8722 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:09.765126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4429373-8b13-4ce6-ba80-57429ac9b529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:09.765750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4429373-8b13-4ce6-ba80-57429ac9b529 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:09.766136 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4429373-8b13-4ce6-ba80-57429ac9b529 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:09.766337 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4429373-8b13-4ce6-ba80-57429ac9b529 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:09.766960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4429373-8b13-4ce6-ba80-57429ac9b529 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:09.768883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4429373-8b13-4ce6-ba80-57429ac9b529 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:09.820420 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54de5ffcf73f4cf69a05c09447e5307e', 'request_id': 'req-77e68458-966a-49cb-800f-88af8d2bf4d2'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:09.827157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54de5ffcf73f4cf69a05c09447e5307e', 'request_id': 'req-77e68458-966a-49cb-800f-88af8d2bf4d2'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:09.833337 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54de5ffcf73f4cf69a05c09447e5307e', 'request_id': 'req-77e68458-966a-49cb-800f-88af8d2bf4d2'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:09.888878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1343/2643] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:09 2022] POST /identity/v3/users => generated 295 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:09.892458 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4429373-8b13-4ce6-ba80-57429ac9b529 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:09.902121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4429373-8b13-4ce6-ba80-57429ac9b529 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:09.924847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] The token cache is being invalidate because project 54de5ffcf73f4cf69a05c09447e5307e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:09.925138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:09.925255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:09.926012 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:09.927771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1301/2644] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:09 2022] DELETE /identity/v3/projects/54de5ffcf73f4cf69a05c09447e5307e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:09.933529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:09.935418 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:09.970191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409a4f4-f9b0-4966-8d0e-cf3874825f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:09.970804 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409a4f4-f9b0-4966-8d0e-cf3874825f1f admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:09.971078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409a4f4-f9b0-4966-8d0e-cf3874825f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:09.971345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2409a4f4-f9b0-4966-8d0e-cf3874825f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:09.971976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409a4f4-f9b0-4966-8d0e-cf3874825f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:09.973761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2409a4f4-f9b0-4966-8d0e-cf3874825f1f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:09.984073 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1344/2645] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:09.993527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409a4f4-f9b0-4966-8d0e-cf3874825f1f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:09.993527 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2409a4f4-f9b0-4966-8d0e-cf3874825f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:10.003921 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2409a4f4-f9b0-4966-8d0e-cf3874825f1f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:10.005106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e68458-966a-49cb-800f-88af8d2bf4d2 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:10.010715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058feeb4-00cf-4777-9b46-c3961d0929a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:10.011347 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058feeb4-00cf-4777-9b46-c3961d0929a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:10.011977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058feeb4-00cf-4777-9b46-c3961d0929a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:10.011977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058feeb4-00cf-4777-9b46-c3961d0929a3 admin admin] PATH_INFO: `/v3/users/b551ea1a13014d2ab27e3b26b3d45474` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:10.013139 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058feeb4-00cf-4777-9b46-c3961d0929a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b551ea1a13014d2ab27e3b26b3d45474)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:10.014905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32013cb-1da3-4606-bc10-16fc02b106cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:10.015235 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058feeb4-00cf-4777-9b46-c3961d0929a3 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:10.016041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32013cb-1da3-4606-bc10-16fc02b106cb admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:10.016558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32013cb-1da3-4606-bc10-16fc02b106cb admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:10.017052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32013cb-1da3-4606-bc10-16fc02b106cb admin admin] PATH_INFO: `/v3/projects/b02d0da5c5e44e9b8b5040dadd5c0a03/users/75fdd0e7384e46279123f83abd67a15f/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:10.034523 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32013cb-1da3-4606-bc10-16fc02b106cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b02d0da5c5e44e9b8b5040dadd5c0a03, user_id=75fdd0e7384e46279123f83abd67a15f, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:10.037683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32013cb-1da3-4606-bc10-16fc02b106cb admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:10.126403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1345/2646] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:09 2022] PUT /identity/v3/projects/b02d0da5c5e44e9b8b5040dadd5c0a03/users/75fdd0e7384e46279123f83abd67a15f/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:10.130234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32013cb-1da3-4606-bc10-16fc02b106cb admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:10.196829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32013cb-1da3-4606-bc10-16fc02b106cb admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:10.203469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35927f1-725b-4708-aece-aca2df1c6dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:10.203469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35927f1-725b-4708-aece-aca2df1c6dbd admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:10.203469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35927f1-725b-4708-aece-aca2df1c6dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:10.203469 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35927f1-725b-4708-aece-aca2df1c6dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:10.203887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35927f1-725b-4708-aece-aca2df1c6dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:10.205083 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35927f1-725b-4708-aece-aca2df1c6dbd admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:10.211781 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-058feeb4-00cf-4777-9b46-c3961d0929a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b551ea1a13014d2ab27e3b26b3d45474'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:10.216610 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1346/2647] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:10.221126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35927f1-725b-4708-aece-aca2df1c6dbd admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:10.229228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-058feeb4-00cf-4777-9b46-c3961d0929a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b551ea1a13014d2ab27e3b26b3d45474'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:10.232916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35927f1-725b-4708-aece-aca2df1c6dbd admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:10.237922 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8ecce7-a8d5-4067-9859-ac5816a31f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:10.238646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ecce7-a8d5-4067-9859-ac5816a31f5d admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:10.239015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ecce7-a8d5-4067-9859-ac5816a31f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:10.239357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8ecce7-a8d5-4067-9859-ac5816a31f5d admin admin] PATH_INFO: `/v3/projects/b02d0da5c5e44e9b8b5040dadd5c0a03/users/75fdd0e7384e46279123f83abd67a15f/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:10.241542 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ecce7-a8d5-4067-9859-ac5816a31f5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b02d0da5c5e44e9b8b5040dadd5c0a03, user_id=75fdd0e7384e46279123f83abd67a15f, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:10.248859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8ecce7-a8d5-4067-9859-ac5816a31f5d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:10.259742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1347/2648] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:10 2022] PUT /identity/v3/projects/b02d0da5c5e44e9b8b5040dadd5c0a03/users/75fdd0e7384e46279123f83abd67a15f/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:10.270019 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46190f33-acd3-49ad-8b88-44bb3afe98d6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:10.270303 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46190f33-acd3-49ad-8b88-44bb3afe98d6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:10.270573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46190f33-acd3-49ad-8b88-44bb3afe98d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:10.298704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-46190f33-acd3-49ad-8b88-44bb3afe98d6 None None] MFA Rules not processed for user `75fdd0e7384e46279123f83abd67a15f`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:10.301210 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-46190f33-acd3-49ad-8b88-44bb3afe98d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:10.329662 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-058feeb4-00cf-4777-9b46-c3961d0929a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b551ea1a13014d2ab27e3b26b3d45474'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:10.331917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-058feeb4-00cf-4777-9b46-c3961d0929a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b551ea1a13014d2ab27e3b26b3d45474'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:10.340370 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-058feeb4-00cf-4777-9b46-c3961d0929a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b551ea1a13014d2ab27e3b26b3d45474'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:10.346477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1302/2649] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:09 2022] DELETE /identity/v3/users/b551ea1a13014d2ab27e3b26b3d45474 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:10.382900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1348/2650] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:10.394556 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058feeb4-00cf-4777-9b46-c3961d0929a3 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:10.396374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-058feeb4-00cf-4777-9b46-c3961d0929a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:10.457970 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058feeb4-00cf-4777-9b46-c3961d0929a3 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:10.462728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d876f-f1b2-4b9f-bb04-9496e69bff59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:10.463449 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d876f-f1b2-4b9f-bb04-9496e69bff59 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:10.463952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d876f-f1b2-4b9f-bb04-9496e69bff59 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:10.464377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d876f-f1b2-4b9f-bb04-9496e69bff59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:10.466339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-316d876f-f1b2-4b9f-bb04-9496e69bff59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:10.528856 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d876f-f1b2-4b9f-bb04-9496e69bff59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:10.530776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d876f-f1b2-4b9f-bb04-9496e69bff59 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:10.541731 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1303/2651] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:10.735620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:10.832936 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:11.091358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46190f33-acd3-49ad-8b88-44bb3afe98d6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:11.092816 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-46190f33-acd3-49ad-8b88-44bb3afe98d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:11.161936 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46190f33-acd3-49ad-8b88-44bb3afe98d6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:11.172638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d876f-f1b2-4b9f-bb04-9496e69bff59 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:11.174132 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-316d876f-f1b2-4b9f-bb04-9496e69bff59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:11.185529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d876f-f1b2-4b9f-bb04-9496e69bff59 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:11.189194 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5259b97-a4ab-426f-a98a-52f070e4696a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:11.189786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5259b97-a4ab-426f-a98a-52f070e4696a service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:11.190118 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5259b97-a4ab-426f-a98a-52f070e4696a service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:11.190387 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5259b97-a4ab-426f-a98a-52f070e4696a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:11.192564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-b5259b97-a4ab-426f-a98a-52f070e4696a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:11.220808 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:11.221447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:11.224074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:11.224074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6 admin admin] PATH_INFO: `/v3/projects/dc13fe1888574bfb81e84328c1bb211d` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:11.224074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc13fe1888574bfb81e84328c1bb211d)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:11.225421 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:11.285085 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5259b97-a4ab-426f-a98a-52f070e4696a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:11.286548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5259b97-a4ab-426f-a98a-52f070e4696a service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:11.296062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1304/2652] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:04:11.377273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc13fe1888574bfb81e84328c1bb211d', 'request_id': 'req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:11.471992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc13fe1888574bfb81e84328c1bb211d', 'request_id': 'req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:11.479495 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc13fe1888574bfb81e84328c1bb211d', 'request_id': 'req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:11.491561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6 admin admin] The token cache is being invalidate because project dc13fe1888574bfb81e84328c1bb211d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:11.491845 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:11.492994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-27f0ed27-b01d-45ef-9ac4-5fe5c8fd90d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:11.494638 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1349/2653] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:11 2022] DELETE /identity/v3/projects/dc13fe1888574bfb81e84328c1bb211d => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:11.499708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe891f-fbe7-4cc0-8a65-35a12aee57a9 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:11.500199 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe891f-fbe7-4cc0-8a65-35a12aee57a9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:11.500331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe891f-fbe7-4cc0-8a65-35a12aee57a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:11.523512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-20fe891f-fbe7-4cc0-8a65-35a12aee57a9 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:11.524755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-20fe891f-fbe7-4cc0-8a65-35a12aee57a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:11.587318 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1305/2654] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:11.602587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6540705-d0f7-48a6-8fbc-db3834d716b8 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:11.602587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6540705-d0f7-48a6-8fbc-db3834d716b8 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:11.602587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6540705-d0f7-48a6-8fbc-db3834d716b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:11.619039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-e6540705-d0f7-48a6-8fbc-db3834d716b8 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:11.620235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-e6540705-d0f7-48a6-8fbc-db3834d716b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:11.628438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1350/2655] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:11.632895 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe891f-fbe7-4cc0-8a65-35a12aee57a9 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:11.644736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe891f-fbe7-4cc0-8a65-35a12aee57a9 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:11.649819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605293c-5161-46cd-b06b-fc996b1b01f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:11.650434 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605293c-5161-46cd-b06b-fc996b1b01f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:11.650710 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605293c-5161-46cd-b06b-fc996b1b01f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:11.650985 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605293c-5161-46cd-b06b-fc996b1b01f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:11.651658 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605293c-5161-46cd-b06b-fc996b1b01f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:04:11.651972 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605293c-5161-46cd-b06b-fc996b1b01f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:11.653655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605293c-5161-46cd-b06b-fc996b1b01f6 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:11.666497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1306/2656] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:04:11.670759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6540705-d0f7-48a6-8fbc-db3834d716b8 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:11.679775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6540705-d0f7-48a6-8fbc-db3834d716b8 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:11.685041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffff20-e872-4b69-8edf-046a7ca04ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:11.685630 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffff20-e872-4b69-8edf-046a7ca04ebc admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:11.685939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffff20-e872-4b69-8edf-046a7ca04ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:11.686212 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fffff20-e872-4b69-8edf-046a7ca04ebc admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:11.686842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fffff20-e872-4b69-8edf-046a7ca04ebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:11.688719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fffff20-e872-4b69-8edf-046a7ca04ebc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:11.833737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1351/2657] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:11 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:11.838461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605293c-5161-46cd-b06b-fc996b1b01f6 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:11.850571 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605293c-5161-46cd-b06b-fc996b1b01f6 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:11.910332 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5537b-864d-4706-bc88-8b63e4c0f543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:11.911034 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5537b-864d-4706-bc88-8b63e4c0f543 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:11.911424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5537b-864d-4706-bc88-8b63e4c0f543 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:11.911720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5537b-864d-4706-bc88-8b63e4c0f543 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:11.912431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5537b-864d-4706-bc88-8b63e4c0f543 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:11.915259 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5537b-864d-4706-bc88-8b63e4c0f543 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:11.998509 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:12.004480 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1307/2658] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:11 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:12.010359 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffff20-e872-4b69-8edf-046a7ca04ebc admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:12.019506 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fffff20-e872-4b69-8edf-046a7ca04ebc admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:12.025425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729e56e-dc31-4fb0-8de3-5a72bfdfede3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:12.026067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729e56e-dc31-4fb0-8de3-5a72bfdfede3 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:12.026348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729e56e-dc31-4fb0-8de3-5a72bfdfede3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:12.026621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729e56e-dc31-4fb0-8de3-5a72bfdfede3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:12.027257 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729e56e-dc31-4fb0-8de3-5a72bfdfede3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:12.028893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9729e56e-dc31-4fb0-8de3-5a72bfdfede3 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:12.039092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1352/2659] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:12.047415 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5537b-864d-4706-bc88-8b63e4c0f543 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:12.057493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5537b-864d-4706-bc88-8b63e4c0f543 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:12.062084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3196f802-ac55-46c5-b152-c063f7a9d15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:12.062921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196f802-ac55-46c5-b152-c063f7a9d15f admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:12.062921 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196f802-ac55-46c5-b152-c063f7a9d15f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:12.063418 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3196f802-ac55-46c5-b152-c063f7a9d15f admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:12.063957 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196f802-ac55-46c5-b152-c063f7a9d15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:12.065589 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3196f802-ac55-46c5-b152-c063f7a9d15f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:12.076583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1308/2660] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:12.080826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729e56e-dc31-4fb0-8de3-5a72bfdfede3 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:12.091497 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729e56e-dc31-4fb0-8de3-5a72bfdfede3 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:12.097957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8631c8de-7286-449c-98f4-d4588a2c1203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:12.098603 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631c8de-7286-449c-98f4-d4588a2c1203 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:12.098878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631c8de-7286-449c-98f4-d4588a2c1203 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:12.099149 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8631c8de-7286-449c-98f4-d4588a2c1203 admin admin] PATH_INFO: `/v3/projects/f01210b7a25646ada544042240c9b7ac/users/ccaf648ec11546098ad41b15b75ad1c7/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:12.114362 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8631c8de-7286-449c-98f4-d4588a2c1203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f01210b7a25646ada544042240c9b7ac, user_id=ccaf648ec11546098ad41b15b75ad1c7, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:12.117053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8631c8de-7286-449c-98f4-d4588a2c1203 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:12.208167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1353/2661] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:12 2022] PUT /identity/v3/projects/f01210b7a25646ada544042240c9b7ac/users/ccaf648ec11546098ad41b15b75ad1c7/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:12.219022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e765c9-392b-498d-9066-4ebab17dbcc2 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:12.219368 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e765c9-392b-498d-9066-4ebab17dbcc2 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:12.219640 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e765c9-392b-498d-9066-4ebab17dbcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:12.250016 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-a8e765c9-392b-498d-9066-4ebab17dbcc2 None None] MFA Rules not processed for user `ccaf648ec11546098ad41b15b75ad1c7`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:12.251123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a8e765c9-392b-498d-9066-4ebab17dbcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:12.318229 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1309/2662] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:12.324124 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8631c8de-7286-449c-98f4-d4588a2c1203 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:12.385490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8631c8de-7286-449c-98f4-d4588a2c1203 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:12.390364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3ad2c-d1a4-4fd4-b58c-23a6312401b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:12.390970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3ad2c-d1a4-4fd4-b58c-23a6312401b6 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:12.391243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3ad2c-d1a4-4fd4-b58c-23a6312401b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:12.391561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b3ad2c-d1a4-4fd4-b58c-23a6312401b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:12.392189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3ad2c-d1a4-4fd4-b58c-23a6312401b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:12.394142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b3ad2c-d1a4-4fd4-b58c-23a6312401b6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:12.490493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:12.498276 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1354/2663] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:12 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:12.502458 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e765c9-392b-498d-9066-4ebab17dbcc2 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:12.518158 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e765c9-392b-498d-9066-4ebab17dbcc2 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:12.583334 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3ad2c-d1a4-4fd4-b58c-23a6312401b6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:12.583737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f72c95-7f93-4ff7-b6b0-94d57eedce06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:12.584434 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f72c95-7f93-4ff7-b6b0-94d57eedce06 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:12.584816 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f72c95-7f93-4ff7-b6b0-94d57eedce06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:12.585160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b5b3ad2c-d1a4-4fd4-b58c-23a6312401b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:12.585287 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f72c95-7f93-4ff7-b6b0-94d57eedce06 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:12.585958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f72c95-7f93-4ff7-b6b0-94d57eedce06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:12.588557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f72c95-7f93-4ff7-b6b0-94d57eedce06 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:12.642806 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b3ad2c-d1a4-4fd4-b58c-23a6312401b6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:12.646449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ab643-fcc3-49bf-b40a-a70b6a248ec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:12.647047 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ab643-fcc3-49bf-b40a-a70b6a248ec1 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:12.647322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ab643-fcc3-49bf-b40a-a70b6a248ec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:12.647583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691ab643-fcc3-49bf-b40a-a70b6a248ec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:12.650020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-691ab643-fcc3-49bf-b40a-a70b6a248ec1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:12.685489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1310/2664] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:12 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:12.689042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f72c95-7f93-4ff7-b6b0-94d57eedce06 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:12.699428 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f72c95-7f93-4ff7-b6b0-94d57eedce06 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:12.704081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b23840-e44e-4e9b-937a-7fd66e56a4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:12.704670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23840-e44e-4e9b-937a-7fd66e56a4ea admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:12.704947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23840-e44e-4e9b-937a-7fd66e56a4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:12.705214 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b23840-e44e-4e9b-937a-7fd66e56a4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:12.705873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b23840-e44e-4e9b-937a-7fd66e56a4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:12.707459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b23840-e44e-4e9b-937a-7fd66e56a4ea admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:12.713295 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ab643-fcc3-49bf-b40a-a70b6a248ec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:12.714743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691ab643-fcc3-49bf-b40a-a70b6a248ec1 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:12.717565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1311/2665] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:12.721347 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b23840-e44e-4e9b-937a-7fd66e56a4ea admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:12.723765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1355/2666] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:04:12.731069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b23840-e44e-4e9b-937a-7fd66e56a4ea admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:12.736328 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7548ec97-020c-409f-ba89-7d1fa2bee196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:12.736613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548ec97-020c-409f-ba89-7d1fa2bee196 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:12.736996 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548ec97-020c-409f-ba89-7d1fa2bee196 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:12.737343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548ec97-020c-409f-ba89-7d1fa2bee196 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:12.738949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548ec97-020c-409f-ba89-7d1fa2bee196 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:12.740192 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7548ec97-020c-409f-ba89-7d1fa2bee196 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:12.751349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1312/2667] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:12.756067 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ab643-fcc3-49bf-b40a-a70b6a248ec1 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:12.771621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691ab643-fcc3-49bf-b40a-a70b6a248ec1 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:12.776704 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbb93c-72a1-415f-97c4-54d3eec5750e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:12.777355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbb93c-72a1-415f-97c4-54d3eec5750e admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:12.777630 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbb93c-72a1-415f-97c4-54d3eec5750e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:12.777946 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbb93c-72a1-415f-97c4-54d3eec5750e admin admin] PATH_INFO: `/v3/projects/7ab23915af014f00bbf82a3254243136/users/9f9f67ea6d7a4c7ba080d8fae1a20ab6/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:12.792888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbb93c-72a1-415f-97c4-54d3eec5750e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ab23915af014f00bbf82a3254243136, user_id=9f9f67ea6d7a4c7ba080d8fae1a20ab6, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:12.798037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dbb93c-72a1-415f-97c4-54d3eec5750e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:12.891373 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1356/2668] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:12 2022] PUT /identity/v3/projects/7ab23915af014f00bbf82a3254243136/users/9f9f67ea6d7a4c7ba080d8fae1a20ab6/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:12.901640 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d1027-3f50-4ee1-9fe0-764232ffac7a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:12.901958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d1027-3f50-4ee1-9fe0-764232ffac7a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:12.902234 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10d1027-3f50-4ee1-9fe0-764232ffac7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:12.906560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:12.933586 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-d10d1027-3f50-4ee1-9fe0-764232ffac7a None None] MFA Rules not processed for user `9f9f67ea6d7a4c7ba080d8fae1a20ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:12.934832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d10d1027-3f50-4ee1-9fe0-764232ffac7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:12.975965 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:13.006749 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1313/2669] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:13.011812 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbb93c-72a1-415f-97c4-54d3eec5750e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:13.077332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dbb93c-72a1-415f-97c4-54d3eec5750e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:13.082465 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4441c-608e-4c5c-ae8a-d4388be33d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:13.083182 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4441c-608e-4c5c-ae8a-d4388be33d7d admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:13.083515 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4441c-608e-4c5c-ae8a-d4388be33d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:13.083874 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4441c-608e-4c5c-ae8a-d4388be33d7d admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:13.084573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4441c-608e-4c5c-ae8a-d4388be33d7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:13.086990 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4441c-608e-4c5c-ae8a-d4388be33d7d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:13.205836 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1357/2670] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:13 2022] POST /identity/v3/projects => generated 387 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:13.210088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d1027-3f50-4ee1-9fe0-764232ffac7a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:13.221144 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10d1027-3f50-4ee1-9fe0-764232ffac7a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:13.280870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1cbfc-c555-4c94-afbc-b9f430a7ca6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:13.281473 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1cbfc-c555-4c94-afbc-b9f430a7ca6a admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:13.281748 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1cbfc-c555-4c94-afbc-b9f430a7ca6a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:13.282053 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f1cbfc-c555-4c94-afbc-b9f430a7ca6a admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:13.282678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1cbfc-c555-4c94-afbc-b9f430a7ca6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:13.284628 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f1cbfc-c555-4c94-afbc-b9f430a7ca6a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:13.392954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1314/2671] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:13 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:13.401708 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4441c-608e-4c5c-ae8a-d4388be33d7d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:13.407516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4441c-608e-4c5c-ae8a-d4388be33d7d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:13.412205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c887b-d8cb-4e50-ad53-2e8a3c9a55fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:13.412797 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c887b-d8cb-4e50-ad53-2e8a3c9a55fb admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:13.413109 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c887b-d8cb-4e50-ad53-2e8a3c9a55fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:13.413378 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c887b-d8cb-4e50-ad53-2e8a3c9a55fb admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:13.414037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c887b-d8cb-4e50-ad53-2e8a3c9a55fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:13.416300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c887b-d8cb-4e50-ad53-2e8a3c9a55fb admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:13.427011 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1358/2672] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:13.431155 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1cbfc-c555-4c94-afbc-b9f430a7ca6a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:13.441300 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f1cbfc-c555-4c94-afbc-b9f430a7ca6a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:13.446553 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c6fcf-9453-4d98-8cd0-4b2a99202341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:13.447237 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c6fcf-9453-4d98-8cd0-4b2a99202341 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:13.447513 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c6fcf-9453-4d98-8cd0-4b2a99202341 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:13.447783 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c6fcf-9453-4d98-8cd0-4b2a99202341 admin admin] PATH_INFO: `/v3/projects/db41d47e82684cbb9617be1645a7e594/users/f2b3bd371dbc4dccbd5a4c45ce305a89/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:13.464764 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c6fcf-9453-4d98-8cd0-4b2a99202341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db41d47e82684cbb9617be1645a7e594, user_id=f2b3bd371dbc4dccbd5a4c45ce305a89, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:13.467526 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c6fcf-9453-4d98-8cd0-4b2a99202341 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:13.581171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1315/2673] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:13 2022] PUT /identity/v3/projects/db41d47e82684cbb9617be1645a7e594/users/f2b3bd371dbc4dccbd5a4c45ce305a89/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:13.585161 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c887b-d8cb-4e50-ad53-2e8a3c9a55fb admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:13.595755 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c887b-d8cb-4e50-ad53-2e8a3c9a55fb admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:13.653878 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0555c-4cb8-428b-aa01-845266ea3eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:13.654529 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0555c-4cb8-428b-aa01-845266ea3eaf admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:13.654806 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0555c-4cb8-428b-aa01-845266ea3eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:13.655094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0555c-4cb8-428b-aa01-845266ea3eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:13.655827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0555c-4cb8-428b-aa01-845266ea3eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:13.657447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0555c-4cb8-428b-aa01-845266ea3eaf admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:13.669530 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1359/2674] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:13.673236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c6fcf-9453-4d98-8cd0-4b2a99202341 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:13.683502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c6fcf-9453-4d98-8cd0-4b2a99202341 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:13.688406 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07488a35-d6c4-4b61-a413-acf355ba4169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:13.689174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07488a35-d6c4-4b61-a413-acf355ba4169 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:13.689455 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07488a35-d6c4-4b61-a413-acf355ba4169 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:13.689731 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07488a35-d6c4-4b61-a413-acf355ba4169 admin admin] PATH_INFO: `/v3/projects/db41d47e82684cbb9617be1645a7e594/users/f2b3bd371dbc4dccbd5a4c45ce305a89/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:13.691668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07488a35-d6c4-4b61-a413-acf355ba4169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db41d47e82684cbb9617be1645a7e594, user_id=f2b3bd371dbc4dccbd5a4c45ce305a89, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:13.694354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07488a35-d6c4-4b61-a413-acf355ba4169 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:13.704757 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1316/2675] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:13 2022] PUT /identity/v3/projects/db41d47e82684cbb9617be1645a7e594/users/f2b3bd371dbc4dccbd5a4c45ce305a89/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:13.716437 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa85a28-c4b2-45bc-a980-c005034a8994 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:13.716766 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa85a28-c4b2-45bc-a980-c005034a8994 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:13.717045 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa85a28-c4b2-45bc-a980-c005034a8994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:13.746325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-9aa85a28-c4b2-45bc-a980-c005034a8994 None None] MFA Rules not processed for user `f2b3bd371dbc4dccbd5a4c45ce305a89`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:13.747937 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-9aa85a28-c4b2-45bc-a980-c005034a8994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:13.818958 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1360/2676] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:13.830307 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07488a35-d6c4-4b61-a413-acf355ba4169 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:13.831982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-07488a35-d6c4-4b61-a413-acf355ba4169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:13.893336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07488a35-d6c4-4b61-a413-acf355ba4169 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:13.897010 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36548850-cc32-4865-8f0b-34eb1dbfbe4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:13.897656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36548850-cc32-4865-8f0b-34eb1dbfbe4b service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:13.897958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36548850-cc32-4865-8f0b-34eb1dbfbe4b service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:13.898233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36548850-cc32-4865-8f0b-34eb1dbfbe4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:13.907753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36548850-cc32-4865-8f0b-34eb1dbfbe4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:13.909279 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36548850-cc32-4865-8f0b-34eb1dbfbe4b service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:13.918638 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1317/2677] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:13.991234 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:14.032245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:14.513734 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa85a28-c4b2-45bc-a980-c005034a8994 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:14.523172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa85a28-c4b2-45bc-a980-c005034a8994 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:14.526777 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8775e-06bc-4440-9be3-964b25aa3c66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:14.527366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8775e-06bc-4440-9be3-964b25aa3c66 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:14.527642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8775e-06bc-4440-9be3-964b25aa3c66 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:14.527912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8775e-06bc-4440-9be3-964b25aa3c66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:14.529877 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-34b8775e-06bc-4440-9be3-964b25aa3c66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:14.591281 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8775e-06bc-4440-9be3-964b25aa3c66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:14.592789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8775e-06bc-4440-9be3-964b25aa3c66 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:14.602355 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1361/2678] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:14.768844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36548850-cc32-4865-8f0b-34eb1dbfbe4b service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:14.770319 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-36548850-cc32-4865-8f0b-34eb1dbfbe4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:14.827615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36548850-cc32-4865-8f0b-34eb1dbfbe4b service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:14.831199 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfb07b-5040-482c-8cd7-86f7e265d95c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:14.831791 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfb07b-5040-482c-8cd7-86f7e265d95c service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:14.832063 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfb07b-5040-482c-8cd7-86f7e265d95c service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:14.832333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfb07b-5040-482c-8cd7-86f7e265d95c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:14.834035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-dacfb07b-5040-482c-8cd7-86f7e265d95c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:14.844170 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfb07b-5040-482c-8cd7-86f7e265d95c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:14.845604 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfb07b-5040-482c-8cd7-86f7e265d95c service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:14.854237 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1318/2679] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:14.936748 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8775e-06bc-4440-9be3-964b25aa3c66 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:14.946868 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8775e-06bc-4440-9be3-964b25aa3c66 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:14.950643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffaf25-05d1-4bd8-8bec-600ad9958ace service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:14.951225 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffaf25-05d1-4bd8-8bec-600ad9958ace service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:14.951500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffaf25-05d1-4bd8-8bec-600ad9958ace service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:14.951768 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fffaf25-05d1-4bd8-8bec-600ad9958ace service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:14.953544 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6fffaf25-05d1-4bd8-8bec-600ad9958ace service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:15.048220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffaf25-05d1-4bd8-8bec-600ad9958ace service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:15.049648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fffaf25-05d1-4bd8-8bec-600ad9958ace service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:15.069426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1362/2680] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:15.411577 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfb07b-5040-482c-8cd7-86f7e265d95c service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:15.420837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfb07b-5040-482c-8cd7-86f7e265d95c service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:15.424432 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a460fd2-a231-4819-8db4-bae4c76b3256 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:15.425072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a460fd2-a231-4819-8db4-bae4c76b3256 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:15.425348 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a460fd2-a231-4819-8db4-bae4c76b3256 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:15.425617 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a460fd2-a231-4819-8db4-bae4c76b3256 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:15.491793 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a460fd2-a231-4819-8db4-bae4c76b3256 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:15.493385 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a460fd2-a231-4819-8db4-bae4c76b3256 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:15.503280 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1319/2681] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:15.538033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:15.905233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:16.381418 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:19.538575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffaf25-05d1-4bd8-8bec-600ad9958ace service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:19.542958 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6fffaf25-05d1-4bd8-8bec-600ad9958ace service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:19.606727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fffaf25-05d1-4bd8-8bec-600ad9958ace service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:19.611153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe46bb-0eb8-470f-a8e3-9b0d5e51f29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:19.611770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe46bb-0eb8-470f-a8e3-9b0d5e51f29e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:19.612041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe46bb-0eb8-470f-a8e3-9b0d5e51f29e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:19.612307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe46bb-0eb8-470f-a8e3-9b0d5e51f29e admin admin] PATH_INFO: `/v3/users/2283370cbffb4188ab6e5f60243e5a37` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:19.613401 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe46bb-0eb8-470f-a8e3-9b0d5e51f29e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2283370cbffb4188ab6e5f60243e5a37)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:19.615369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe46bb-0eb8-470f-a8e3-9b0d5e51f29e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:19.878621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3afe46bb-0eb8-470f-a8e3-9b0d5e51f29e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2283370cbffb4188ab6e5f60243e5a37'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:19.974679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3afe46bb-0eb8-470f-a8e3-9b0d5e51f29e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2283370cbffb4188ab6e5f60243e5a37'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:19.988893 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3afe46bb-0eb8-470f-a8e3-9b0d5e51f29e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2283370cbffb4188ab6e5f60243e5a37'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:19.989374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3afe46bb-0eb8-470f-a8e3-9b0d5e51f29e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2283370cbffb4188ab6e5f60243e5a37'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:19.990174 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-3afe46bb-0eb8-470f-a8e3-9b0d5e51f29e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2283370cbffb4188ab6e5f60243e5a37'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:20.003432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1363/2682] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:19 2022] DELETE /identity/v3/users/2283370cbffb4188ab6e5f60243e5a37 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:20.008088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5039d2-a6f1-42fa-b913-1d98cd049b88 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:20.010741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5039d2-a6f1-42fa-b913-1d98cd049b88 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:20.010741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5039d2-a6f1-42fa-b913-1d98cd049b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:20.023270 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:20.029542 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-9e5039d2-a6f1-42fa-b913-1d98cd049b88 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:20.030792 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-9e5039d2-a6f1-42fa-b913-1d98cd049b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:20.088772 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1320/2683] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:20.098543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe46bb-0eb8-470f-a8e3-9b0d5e51f29e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:20.102360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3afe46bb-0eb8-470f-a8e3-9b0d5e51f29e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:20.114767 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:20.158548 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe46bb-0eb8-470f-a8e3-9b0d5e51f29e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:20.162290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a3e33-e9f3-48cf-b71e-d041183a7622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:20.162896 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a3e33-e9f3-48cf-b71e-d041183a7622 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:20.163174 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a3e33-e9f3-48cf-b71e-d041183a7622 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:20.163450 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3a3e33-e9f3-48cf-b71e-d041183a7622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:20.173094 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a3e33-e9f3-48cf-b71e-d041183a7622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:20.174562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3a3e33-e9f3-48cf-b71e-d041183a7622 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:20.184032 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1364/2684] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:04:20.472372 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5039d2-a6f1-42fa-b913-1d98cd049b88 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:20.474407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-9e5039d2-a6f1-42fa-b913-1d98cd049b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:20.541831 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5039d2-a6f1-42fa-b913-1d98cd049b88 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:20.545412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ec944-23fe-4f7e-98a0-666ba2f2e5f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:20.546071 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ec944-23fe-4f7e-98a0-666ba2f2e5f2 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:20.546349 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ec944-23fe-4f7e-98a0-666ba2f2e5f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:20.546624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ec944-23fe-4f7e-98a0-666ba2f2e5f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:20.548647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e66ec944-23fe-4f7e-98a0-666ba2f2e5f2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:20.610281 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ec944-23fe-4f7e-98a0-666ba2f2e5f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:20.611723 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ec944-23fe-4f7e-98a0-666ba2f2e5f2 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:20.621270 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1321/2685] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:20.638695 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a3e33-e9f3-48cf-b71e-d041183a7622 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:20.641430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3d3a3e33-e9f3-48cf-b71e-d041183a7622 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:20.657001 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3a3e33-e9f3-48cf-b71e-d041183a7622 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:20.662898 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:20.663829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:20.664166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:20.664508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] PATH_INFO: `/v3/projects/ddd31ffab3ea4564aa7b27b1b3ccf95d` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:20.666479 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddd31ffab3ea4564aa7b27b1b3ccf95d)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:20.669227 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:20.784924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddd31ffab3ea4564aa7b27b1b3ccf95d', 'request_id': 'req-7312034b-a6f7-4031-b82c-8bf3541f423c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:20.841815 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddd31ffab3ea4564aa7b27b1b3ccf95d', 'request_id': 'req-7312034b-a6f7-4031-b82c-8bf3541f423c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:20.848513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddd31ffab3ea4564aa7b27b1b3ccf95d', 'request_id': 'req-7312034b-a6f7-4031-b82c-8bf3541f423c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:20.859636 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] The token cache is being invalidate because project ddd31ffab3ea4564aa7b27b1b3ccf95d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:20.859952 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:20.860599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:20.862109 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1365/2686] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:20 2022] DELETE /identity/v3/projects/ddd31ffab3ea4564aa7b27b1b3ccf95d => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:20.866110 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ec944-23fe-4f7e-98a0-666ba2f2e5f2 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:20.877502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ec944-23fe-4f7e-98a0-666ba2f2e5f2 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:20.934860 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57ac1a-5755-4d22-8530-d373d0d3da1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:20.935508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57ac1a-5755-4d22-8530-d373d0d3da1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:20.935789 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57ac1a-5755-4d22-8530-d373d0d3da1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:20.936062 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b57ac1a-5755-4d22-8530-d373d0d3da1b admin admin] PATH_INFO: `/v3/users/cdf79a7aee3a42c18c90796e5f0c0c60` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:20.937204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57ac1a-5755-4d22-8530-d373d0d3da1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdf79a7aee3a42c18c90796e5f0c0c60)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:20.939169 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b57ac1a-5755-4d22-8530-d373d0d3da1b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:20.946239 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:21.154910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5b57ac1a-5755-4d22-8530-d373d0d3da1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdf79a7aee3a42c18c90796e5f0c0c60'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:21.167218 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5b57ac1a-5755-4d22-8530-d373d0d3da1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdf79a7aee3a42c18c90796e5f0c0c60'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:21.277885 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5b57ac1a-5755-4d22-8530-d373d0d3da1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdf79a7aee3a42c18c90796e5f0c0c60'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:21.278562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5b57ac1a-5755-4d22-8530-d373d0d3da1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdf79a7aee3a42c18c90796e5f0c0c60'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:21.286044 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-5b57ac1a-5755-4d22-8530-d373d0d3da1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdf79a7aee3a42c18c90796e5f0c0c60'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:21.292003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1322/2687] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:20 2022] DELETE /identity/v3/users/cdf79a7aee3a42c18c90796e5f0c0c60 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:21.740726 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:21.742211 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:21.758876 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7312034b-a6f7-4031-b82c-8bf3541f423c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:21.818589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:21.819369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:21.819711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:21.820185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] PATH_INFO: `/v3/projects/64d2388928f5436eb2c35be00a28e06e` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:21.822133 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64d2388928f5436eb2c35be00a28e06e)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:21.832997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:21.964510 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:21.970712 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64d2388928f5436eb2c35be00a28e06e', 'request_id': 'req-6f7b9751-36f4-4a2d-b087-e717b59db922'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:22.089503 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64d2388928f5436eb2c35be00a28e06e', 'request_id': 'req-6f7b9751-36f4-4a2d-b087-e717b59db922'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:22.096628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64d2388928f5436eb2c35be00a28e06e', 'request_id': 'req-6f7b9751-36f4-4a2d-b087-e717b59db922'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:22.107566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] The token cache is being invalidate because project 64d2388928f5436eb2c35be00a28e06e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:22.107859 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:22.108505 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:22.110051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1366/2688] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:21 2022] DELETE /identity/v3/projects/64d2388928f5436eb2c35be00a28e06e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:22.114254 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57ac1a-5755-4d22-8530-d373d0d3da1b admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:22.115668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-5b57ac1a-5755-4d22-8530-d373d0d3da1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:22.128871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b57ac1a-5755-4d22-8530-d373d0d3da1b admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:22.193644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf6c5c-76f0-49e4-8b62-dfc3b2cc2280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:22.194328 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf6c5c-76f0-49e4-8b62-dfc3b2cc2280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:22.194580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf6c5c-76f0-49e4-8b62-dfc3b2cc2280 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:22.194853 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf6c5c-76f0-49e4-8b62-dfc3b2cc2280 admin admin] PATH_INFO: `/v3/users/75fdd0e7384e46279123f83abd67a15f` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:22.195974 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf6c5c-76f0-49e4-8b62-dfc3b2cc2280 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75fdd0e7384e46279123f83abd67a15f)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:22.198097 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf6c5c-76f0-49e4-8b62-dfc3b2cc2280 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:22.398113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-55bf6c5c-76f0-49e4-8b62-dfc3b2cc2280 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75fdd0e7384e46279123f83abd67a15f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:22.411133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-55bf6c5c-76f0-49e4-8b62-dfc3b2cc2280 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75fdd0e7384e46279123f83abd67a15f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:22.518040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-55bf6c5c-76f0-49e4-8b62-dfc3b2cc2280 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75fdd0e7384e46279123f83abd67a15f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:22.518658 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-55bf6c5c-76f0-49e4-8b62-dfc3b2cc2280 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75fdd0e7384e46279123f83abd67a15f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:22.526926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-55bf6c5c-76f0-49e4-8b62-dfc3b2cc2280 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75fdd0e7384e46279123f83abd67a15f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:22.529037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:22.533122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1323/2689] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:22 2022] DELETE /identity/v3/users/75fdd0e7384e46279123f83abd67a15f => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:22.993942 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:22.995581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:23.008608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b9751-36f4-4a2d-b087-e717b59db922 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:23.091171 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0de28-37aa-466d-823d-31a2d27c9cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:23.091807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0de28-37aa-466d-823d-31a2d27c9cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:23.092132 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0de28-37aa-466d-823d-31a2d27c9cee admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:23.092455 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0de28-37aa-466d-823d-31a2d27c9cee admin admin] PATH_INFO: `/v3/projects/b02d0da5c5e44e9b8b5040dadd5c0a03` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:23.093555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0de28-37aa-466d-823d-31a2d27c9cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=b02d0da5c5e44e9b8b5040dadd5c0a03)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:23.095487 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0de28-37aa-466d-823d-31a2d27c9cee admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:23.244346 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-11a0de28-37aa-466d-823d-31a2d27c9cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b02d0da5c5e44e9b8b5040dadd5c0a03', 'request_id': 'req-11a0de28-37aa-466d-823d-31a2d27c9cee'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:23.364126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-11a0de28-37aa-466d-823d-31a2d27c9cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b02d0da5c5e44e9b8b5040dadd5c0a03', 'request_id': 'req-11a0de28-37aa-466d-823d-31a2d27c9cee'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:23.375906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-11a0de28-37aa-466d-823d-31a2d27c9cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b02d0da5c5e44e9b8b5040dadd5c0a03', 'request_id': 'req-11a0de28-37aa-466d-823d-31a2d27c9cee'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:23.382491 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-11a0de28-37aa-466d-823d-31a2d27c9cee admin admin] The token cache is being invalidate because project b02d0da5c5e44e9b8b5040dadd5c0a03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:23.382773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-11a0de28-37aa-466d-823d-31a2d27c9cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:23.383404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-11a0de28-37aa-466d-823d-31a2d27c9cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:23.385561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1367/2690] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:22 2022] DELETE /identity/v3/projects/b02d0da5c5e44e9b8b5040dadd5c0a03 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:24.852800 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf6c5c-76f0-49e4-8b62-dfc3b2cc2280 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:24.854482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-55bf6c5c-76f0-49e4-8b62-dfc3b2cc2280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:24.867659 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf6c5c-76f0-49e4-8b62-dfc3b2cc2280 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:24.928336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e24fa-5d48-4792-9ea6-e4b51c149c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:24.928962 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e24fa-5d48-4792-9ea6-e4b51c149c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:24.929249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e24fa-5d48-4792-9ea6-e4b51c149c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:24.929506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e24fa-5d48-4792-9ea6-e4b51c149c61 admin admin] PATH_INFO: `/v3/users/ccaf648ec11546098ad41b15b75ad1c7` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:24.930633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e24fa-5d48-4792-9ea6-e4b51c149c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccaf648ec11546098ad41b15b75ad1c7)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:24.933080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e24fa-5d48-4792-9ea6-e4b51c149c61 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:25.166102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6f0e24fa-5d48-4792-9ea6-e4b51c149c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccaf648ec11546098ad41b15b75ad1c7'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:25.178569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6f0e24fa-5d48-4792-9ea6-e4b51c149c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccaf648ec11546098ad41b15b75ad1c7'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:25.232179 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6f0e24fa-5d48-4792-9ea6-e4b51c149c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccaf648ec11546098ad41b15b75ad1c7'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:25.232827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6f0e24fa-5d48-4792-9ea6-e4b51c149c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccaf648ec11546098ad41b15b75ad1c7'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:25.240989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-6f0e24fa-5d48-4792-9ea6-e4b51c149c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccaf648ec11546098ad41b15b75ad1c7'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:25.252464 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1324/2691] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:24 2022] DELETE /identity/v3/users/ccaf648ec11546098ad41b15b75ad1c7 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:25.255585 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd31157-4e5f-4922-a098-18c81fd7f62a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:25.256081 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd31157-4e5f-4922-a098-18c81fd7f62a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:25.256182 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd31157-4e5f-4922-a098-18c81fd7f62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:25.274787 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-5fd31157-4e5f-4922-a098-18c81fd7f62a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:25.276001 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5fd31157-4e5f-4922-a098-18c81fd7f62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:25.331166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:25.344427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1368/2692] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:25.350877 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e24fa-5d48-4792-9ea6-e4b51c149c61 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:25.352328 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6f0e24fa-5d48-4792-9ea6-e4b51c149c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:25.414481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e24fa-5d48-4792-9ea6-e4b51c149c61 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:25.419568 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d58d3-2833-4e28-829b-cc78ed593fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:25.420255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d58d3-2833-4e28-829b-cc78ed593fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:25.420601 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d58d3-2833-4e28-829b-cc78ed593fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:25.420939 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d58d3-2833-4e28-829b-cc78ed593fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:25.432937 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d58d3-2833-4e28-829b-cc78ed593fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:25.436040 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d58d3-2833-4e28-829b-cc78ed593fd2 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:25.446469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1325/2693] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:25.878897 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd31157-4e5f-4922-a098-18c81fd7f62a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:25.880325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-5fd31157-4e5f-4922-a098-18c81fd7f62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:25.898085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd31157-4e5f-4922-a098-18c81fd7f62a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:25.903735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:25.904367 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:25.904640 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:25.904912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] PATH_INFO: `/v3/projects/f01210b7a25646ada544042240c9b7ac` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:25.906077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f01210b7a25646ada544042240c9b7ac)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:25.907997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:26.048408 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f01210b7a25646ada544042240c9b7ac', 'request_id': 'req-bbcd8305-d81d-4b41-8d97-5c10057f036d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:26.127117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f01210b7a25646ada544042240c9b7ac', 'request_id': 'req-bbcd8305-d81d-4b41-8d97-5c10057f036d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:26.133510 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f01210b7a25646ada544042240c9b7ac', 'request_id': 'req-bbcd8305-d81d-4b41-8d97-5c10057f036d'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:26.144275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] The token cache is being invalidate because project f01210b7a25646ada544042240c9b7ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:26.144556 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:26.145167 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:26.146695 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1369/2694] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:25 2022] DELETE /identity/v3/projects/f01210b7a25646ada544042240c9b7ac => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:26.151468 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d58d3-2833-4e28-829b-cc78ed593fd2 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:26.161245 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d58d3-2833-4e28-829b-cc78ed593fd2 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:26.179093 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:26.216888 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5d526-ca0b-4e1b-a160-1396f85befe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:26.217528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5d526-ca0b-4e1b-a160-1396f85befe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:26.217823 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5d526-ca0b-4e1b-a160-1396f85befe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:26.218088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5d526-ca0b-4e1b-a160-1396f85befe6 admin admin] PATH_INFO: `/v3/users/9f9f67ea6d7a4c7ba080d8fae1a20ab6` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:26.219182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5d526-ca0b-4e1b-a160-1396f85befe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f9f67ea6d7a4c7ba080d8fae1a20ab6)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:26.221081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5d526-ca0b-4e1b-a160-1396f85befe6 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:26.410830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-2ff5d526-ca0b-4e1b-a160-1396f85befe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f9f67ea6d7a4c7ba080d8fae1a20ab6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:26.424146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-2ff5d526-ca0b-4e1b-a160-1396f85befe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f9f67ea6d7a4c7ba080d8fae1a20ab6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:26.523329 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-2ff5d526-ca0b-4e1b-a160-1396f85befe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f9f67ea6d7a4c7ba080d8fae1a20ab6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:26.523952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-2ff5d526-ca0b-4e1b-a160-1396f85befe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f9f67ea6d7a4c7ba080d8fae1a20ab6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:26.531633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-2ff5d526-ca0b-4e1b-a160-1396f85befe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f9f67ea6d7a4c7ba080d8fae1a20ab6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:26.538331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1326/2695] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:26 2022] DELETE /identity/v3/users/9f9f67ea6d7a4c7ba080d8fae1a20ab6 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:27.021841 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:27.023378 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:27.040077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd8305-d81d-4b41-8d97-5c10057f036d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:27.105078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:27.110007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:27.110291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:27.110566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] PATH_INFO: `/v3/projects/7ab23915af014f00bbf82a3254243136` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:27.111798 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ab23915af014f00bbf82a3254243136)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:27.113658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:27.272958 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ab23915af014f00bbf82a3254243136', 'request_id': 'req-c5de5090-e263-4376-81d5-bc36de4d428e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:27.380670 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ab23915af014f00bbf82a3254243136', 'request_id': 'req-c5de5090-e263-4376-81d5-bc36de4d428e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:27.388391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ab23915af014f00bbf82a3254243136', 'request_id': 'req-c5de5090-e263-4376-81d5-bc36de4d428e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:27.401303 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] The token cache is being invalidate because project 7ab23915af014f00bbf82a3254243136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:27.401594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:27.402447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:27.404154 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1370/2696] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:27 2022] DELETE /identity/v3/projects/7ab23915af014f00bbf82a3254243136 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:27.408185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5d526-ca0b-4e1b-a160-1396f85befe6 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:27.410013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2ff5d526-ca0b-4e1b-a160-1396f85befe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:27.424378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5d526-ca0b-4e1b-a160-1396f85befe6 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:27.482725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc4020-1ced-4585-a5b3-9cd2f90e3be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:27.483512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc4020-1ced-4585-a5b3-9cd2f90e3be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:27.483799 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc4020-1ced-4585-a5b3-9cd2f90e3be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:27.484098 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bc4020-1ced-4585-a5b3-9cd2f90e3be7 admin admin] PATH_INFO: `/v3/users/f2b3bd371dbc4dccbd5a4c45ce305a89` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:27.485297 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc4020-1ced-4585-a5b3-9cd2f90e3be7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2b3bd371dbc4dccbd5a4c45ce305a89)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:27.487267 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bc4020-1ced-4585-a5b3-9cd2f90e3be7 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:27.644716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:27.647304 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-52bc4020-1ced-4585-a5b3-9cd2f90e3be7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2b3bd371dbc4dccbd5a4c45ce305a89'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:27.659003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-52bc4020-1ced-4585-a5b3-9cd2f90e3be7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2b3bd371dbc4dccbd5a4c45ce305a89'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:27.758846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-52bc4020-1ced-4585-a5b3-9cd2f90e3be7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2b3bd371dbc4dccbd5a4c45ce305a89'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:27.759711 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-52bc4020-1ced-4585-a5b3-9cd2f90e3be7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2b3bd371dbc4dccbd5a4c45ce305a89'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:27.782497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-52bc4020-1ced-4585-a5b3-9cd2f90e3be7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2b3bd371dbc4dccbd5a4c45ce305a89'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:27.796919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1327/2697] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:27 2022] DELETE /identity/v3/users/f2b3bd371dbc4dccbd5a4c45ce305a89 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:28.348880 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:28.350717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:28.371109 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de5090-e263-4376-81d5-bc36de4d428e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:28.391130 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:28.448476 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:28.449345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:28.449683 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:28.450202 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc admin admin] PATH_INFO: `/v3/projects/db41d47e82684cbb9617be1645a7e594` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:28.451413 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=db41d47e82684cbb9617be1645a7e594)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:28.453696 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:28.574597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc4020-1ced-4585-a5b3-9cd2f90e3be7 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:28.576111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-52bc4020-1ced-4585-a5b3-9cd2f90e3be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:28.591687 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bc4020-1ced-4585-a5b3-9cd2f90e3be7 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:28.598288 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db41d47e82684cbb9617be1645a7e594', 'request_id': 'req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:28.648562 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f5a11-2eac-4877-b7e1-576499472176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:28.649187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f5a11-2eac-4877-b7e1-576499472176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:28.649491 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f5a11-2eac-4877-b7e1-576499472176 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:28.649762 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f5a11-2eac-4877-b7e1-576499472176 admin admin] PATH_INFO: `/v3/users/41e5355a524b4e50885519a71c11530c` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:28.650942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f5a11-2eac-4877-b7e1-576499472176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41e5355a524b4e50885519a71c11530c)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:28.652840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f5a11-2eac-4877-b7e1-576499472176 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:28.676146 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db41d47e82684cbb9617be1645a7e594', 'request_id': 'req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:28.683431 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db41d47e82684cbb9617be1645a7e594', 'request_id': 'req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:28.694034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc admin admin] The token cache is being invalidate because project db41d47e82684cbb9617be1645a7e594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:28.694325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:28.694957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bf98ba1a-a425-49f8-a0bb-4a9eb76d54bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:28.696457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1371/2698] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:28 2022] DELETE /identity/v3/projects/db41d47e82684cbb9617be1645a7e594 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:28.702673 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fc427-c9f4-48d8-bfcd-051de91c83af None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:28.702950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fc427-c9f4-48d8-bfcd-051de91c83af None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:28.703221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fc427-c9f4-48d8-bfcd-051de91c83af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:28.722502 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a05fc427-c9f4-48d8-bfcd-051de91c83af None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:28.723693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a05fc427-c9f4-48d8-bfcd-051de91c83af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:28.732126 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1372/2699] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:28.740057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b2a43-5afe-4172-b5c0-203ed3d76de2 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:28.740332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b2a43-5afe-4172-b5c0-203ed3d76de2 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:28.740601 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b2a43-5afe-4172-b5c0-203ed3d76de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:28.759733 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-6c4b2a43-5afe-4172-b5c0-203ed3d76de2 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:28.760890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6c4b2a43-5afe-4172-b5c0-203ed3d76de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:28.769119 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1373/2700] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:28.772973 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b2a43-5afe-4172-b5c0-203ed3d76de2 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:28.781668 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b2a43-5afe-4172-b5c0-203ed3d76de2 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:28.786230 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3009f-8963-4c34-9354-ecca00539774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:28.786779 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3009f-8963-4c34-9354-ecca00539774 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:28.787051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3009f-8963-4c34-9354-ecca00539774 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:28.787315 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3009f-8963-4c34-9354-ecca00539774 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:28.788153 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3009f-8963-4c34-9354-ecca00539774 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:04:28.788487 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3009f-8963-4c34-9354-ecca00539774 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:28.790072 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea3009f-8963-4c34-9354-ecca00539774 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:28.801903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1374/2701] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 20:04:28.805751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3009f-8963-4c34-9354-ecca00539774 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:28.816053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3009f-8963-4c34-9354-ecca00539774 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:28.818023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-500f5a11-2eac-4877-b7e1-576499472176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41e5355a524b4e50885519a71c11530c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:28.820839 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43f05b-9f7d-4a94-90fd-0f33f65ca43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:28.821353 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43f05b-9f7d-4a94-90fd-0f33f65ca43b admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:28.821631 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43f05b-9f7d-4a94-90fd-0f33f65ca43b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:28.821933 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43f05b-9f7d-4a94-90fd-0f33f65ca43b admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:28.822538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43f05b-9f7d-4a94-90fd-0f33f65ca43b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:28.824389 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43f05b-9f7d-4a94-90fd-0f33f65ca43b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:28.830296 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-500f5a11-2eac-4877-b7e1-576499472176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41e5355a524b4e50885519a71c11530c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:28.910512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-500f5a11-2eac-4877-b7e1-576499472176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41e5355a524b4e50885519a71c11530c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:28.911257 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-500f5a11-2eac-4877-b7e1-576499472176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41e5355a524b4e50885519a71c11530c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:28.918273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1375/2702] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:28 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:28.919498 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-500f5a11-2eac-4877-b7e1-576499472176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41e5355a524b4e50885519a71c11530c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:28.922385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43f05b-9f7d-4a94-90fd-0f33f65ca43b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:28.923979 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-cc43f05b-9f7d-4a94-90fd-0f33f65ca43b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:28.925563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1328/2703] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:28 2022] DELETE /identity/v3/users/41e5355a524b4e50885519a71c11530c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:28.990916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43f05b-9f7d-4a94-90fd-0f33f65ca43b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:29.050972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21c348-376e-4e15-8ebb-8d717474d37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:29.051664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21c348-376e-4e15-8ebb-8d717474d37b admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:29.052062 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21c348-376e-4e15-8ebb-8d717474d37b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:29.052403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21c348-376e-4e15-8ebb-8d717474d37b admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:29.053096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21c348-376e-4e15-8ebb-8d717474d37b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:29.055114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21c348-376e-4e15-8ebb-8d717474d37b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:29.155290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1376/2704] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:28 2022] POST /identity/v3/users => generated 289 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:29.156406 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f5a11-2eac-4877-b7e1-576499472176 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:29.166466 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f5a11-2eac-4877-b7e1-576499472176 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:29.171413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1f4b0-4458-457a-a0dc-3b5c1a246250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:29.172092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1f4b0-4458-457a-a0dc-3b5c1a246250 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:29.172469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1f4b0-4458-457a-a0dc-3b5c1a246250 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:29.172819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de1f4b0-4458-457a-a0dc-3b5c1a246250 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:29.173578 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1f4b0-4458-457a-a0dc-3b5c1a246250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:29.175271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de1f4b0-4458-457a-a0dc-3b5c1a246250 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:29.186343 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1329/2705] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:29.190128 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21c348-376e-4e15-8ebb-8d717474d37b admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:29.200797 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21c348-376e-4e15-8ebb-8d717474d37b admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:29.205876 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde4d6b-37ed-49c1-bfef-960fb24d1688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:29.206477 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde4d6b-37ed-49c1-bfef-960fb24d1688 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:29.206753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde4d6b-37ed-49c1-bfef-960fb24d1688 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:29.207025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde4d6b-37ed-49c1-bfef-960fb24d1688 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:29.207655 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde4d6b-37ed-49c1-bfef-960fb24d1688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:29.209245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde4d6b-37ed-49c1-bfef-960fb24d1688 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:29.219388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1377/2706] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:29.232362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1f4b0-4458-457a-a0dc-3b5c1a246250 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:29.242280 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de1f4b0-4458-457a-a0dc-3b5c1a246250 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:29.247461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619d2c7-0226-4bf1-a398-014e6788b236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:29.248121 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619d2c7-0226-4bf1-a398-014e6788b236 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:29.248424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619d2c7-0226-4bf1-a398-014e6788b236 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:29.248720 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619d2c7-0226-4bf1-a398-014e6788b236 admin admin] PATH_INFO: `/v3/projects/291c212967264fbfbb6dd51b92eb5220/users/b19ac61d48034c0cae49e77759d646e5/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:29.264727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619d2c7-0226-4bf1-a398-014e6788b236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=291c212967264fbfbb6dd51b92eb5220, user_id=b19ac61d48034c0cae49e77759d646e5, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:29.267645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619d2c7-0226-4bf1-a398-014e6788b236 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:29.272476 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde4d6b-37ed-49c1-bfef-960fb24d1688 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:29.274365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fdde4d6b-37ed-49c1-bfef-960fb24d1688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:29.334061 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde4d6b-37ed-49c1-bfef-960fb24d1688 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:29.334061 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:29.334061 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:29.334061 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:29.334952 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] PATH_INFO: `/v3/projects/29de914be2644251b09595f0c20abb7f` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:29.334952 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29de914be2644251b09595f0c20abb7f)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:29.334952 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:29.426529 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1330/2707] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:29 2022] PUT /identity/v3/projects/291c212967264fbfbb6dd51b92eb5220/users/b19ac61d48034c0cae49e77759d646e5/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:29.438983 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e24cb-3f77-4834-8f81-752c4ec11ce6 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:29.439274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e24cb-3f77-4834-8f81-752c4ec11ce6 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:29.439531 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e24cb-3f77-4834-8f81-752c4ec11ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:29.448647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:29.450761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:29.467772 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-df2e24cb-3f77-4834-8f81-752c4ec11ce6 None None] MFA Rules not processed for user `b19ac61d48034c0cae49e77759d646e5`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:29.468566 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29de914be2644251b09595f0c20abb7f', 'request_id': 'req-56e8b16d-5515-4650-b00f-46e4ddf51465'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:29.468971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-df2e24cb-3f77-4834-8f81-752c4ec11ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:29.528926 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29de914be2644251b09595f0c20abb7f', 'request_id': 'req-56e8b16d-5515-4650-b00f-46e4ddf51465'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:29.533579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1331/2708] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:29.536329 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29de914be2644251b09595f0c20abb7f', 'request_id': 'req-56e8b16d-5515-4650-b00f-46e4ddf51465'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:29.538023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e24cb-3f77-4834-8f81-752c4ec11ce6 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:29.547364 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] The token cache is being invalidate because project 29de914be2644251b09595f0c20abb7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:29.547633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:29.548292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:29.549776 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1378/2709] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:29 2022] DELETE /identity/v3/projects/29de914be2644251b09595f0c20abb7f => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:29.553528 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:29.554850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:29.602295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e24cb-3f77-4834-8f81-752c4ec11ce6 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:29.607478 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541e381-479e-4f92-a65c-53db0b8198b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:29.608036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e381-479e-4f92-a65c-53db0b8198b4 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:29.608310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e381-479e-4f92-a65c-53db0b8198b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:29.608603 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e381-479e-4f92-a65c-53db0b8198b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:29.609234 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541e381-479e-4f92-a65c-53db0b8198b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:29.611169 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541e381-479e-4f92-a65c-53db0b8198b4 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:29.622298 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8b16d-5515-4650-b00f-46e4ddf51465 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:29.630130 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fbf8e3-2d79-4ad2-9420-c011616f6489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:29.630811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbf8e3-2d79-4ad2-9420-c011616f6489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:29.631085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbf8e3-2d79-4ad2-9420-c011616f6489 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:29.633121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fbf8e3-2d79-4ad2-9420-c011616f6489 admin admin] PATH_INFO: `/v3/users/bd8613650a1147f3ac9d008b6591cdf8` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:29.633121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fbf8e3-2d79-4ad2-9420-c011616f6489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd8613650a1147f3ac9d008b6591cdf8)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:29.634706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fbf8e3-2d79-4ad2-9420-c011616f6489 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:29.679564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1332/2710] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:29 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:29.684081 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541e381-479e-4f92-a65c-53db0b8198b4 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:29.685694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0541e381-479e-4f92-a65c-53db0b8198b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:29.749173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541e381-479e-4f92-a65c-53db0b8198b4 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:29.754070 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1368e3-ddc5-4c40-9c23-d5299bc05200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:29.754680 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1368e3-ddc5-4c40-9c23-d5299bc05200 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:29.754954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1368e3-ddc5-4c40-9c23-d5299bc05200 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:29.755221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1368e3-ddc5-4c40-9c23-d5299bc05200 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:29.755843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1368e3-ddc5-4c40-9c23-d5299bc05200 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:29.757736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1368e3-ddc5-4c40-9c23-d5299bc05200 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:29.827516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84fbf8e3-2d79-4ad2-9420-c011616f6489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd8613650a1147f3ac9d008b6591cdf8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:29.845683 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1333/2711] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:29 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:29.850540 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1368e3-ddc5-4c40-9c23-d5299bc05200 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:29.865339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1368e3-ddc5-4c40-9c23-d5299bc05200 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:29.870797 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77916c21-4c36-44ed-9050-8c6ee88b0649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:29.871620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77916c21-4c36-44ed-9050-8c6ee88b0649 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:29.871818 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77916c21-4c36-44ed-9050-8c6ee88b0649 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:29.872084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77916c21-4c36-44ed-9050-8c6ee88b0649 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:29.872725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77916c21-4c36-44ed-9050-8c6ee88b0649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:29.874373 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77916c21-4c36-44ed-9050-8c6ee88b0649 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:29.887557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1334/2712] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:29.891534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77916c21-4c36-44ed-9050-8c6ee88b0649 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:29.901625 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77916c21-4c36-44ed-9050-8c6ee88b0649 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:29.906656 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb32a2-3780-4484-9881-2a998cd5e80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:29.907383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb32a2-3780-4484-9881-2a998cd5e80a admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:29.907668 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb32a2-3780-4484-9881-2a998cd5e80a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:29.907958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb32a2-3780-4484-9881-2a998cd5e80a admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:29.908596 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb32a2-3780-4484-9881-2a998cd5e80a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:29.910262 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb32a2-3780-4484-9881-2a998cd5e80a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:29.921725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1335/2713] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:29.925777 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb32a2-3780-4484-9881-2a998cd5e80a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:29.935333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb32a2-3780-4484-9881-2a998cd5e80a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:29.939931 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91cd20-de7d-4ae7-9678-073af5359522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:29.940549 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91cd20-de7d-4ae7-9678-073af5359522 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:29.940828 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91cd20-de7d-4ae7-9678-073af5359522 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:29.941102 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91cd20-de7d-4ae7-9678-073af5359522 admin admin] PATH_INFO: `/v3/projects/32dd354712424871a0da39582ef97b91/users/ad0ed8c109cf4c0a965ad26bcdfa4f49/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:29.946613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84fbf8e3-2d79-4ad2-9420-c011616f6489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd8613650a1147f3ac9d008b6591cdf8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:29.955392 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91cd20-de7d-4ae7-9678-073af5359522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32dd354712424871a0da39582ef97b91, user_id=ad0ed8c109cf4c0a965ad26bcdfa4f49, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:29.965239 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a91cd20-de7d-4ae7-9678-073af5359522 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:29.965363 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84fbf8e3-2d79-4ad2-9420-c011616f6489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd8613650a1147f3ac9d008b6591cdf8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:29.965363 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84fbf8e3-2d79-4ad2-9420-c011616f6489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd8613650a1147f3ac9d008b6591cdf8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:29.965363 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-84fbf8e3-2d79-4ad2-9420-c011616f6489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd8613650a1147f3ac9d008b6591cdf8'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:29.974791 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1379/2714] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:29 2022] DELETE /identity/v3/users/bd8613650a1147f3ac9d008b6591cdf8 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:29.987070 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:30.049904 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1336/2715] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:29 2022] PUT /identity/v3/projects/32dd354712424871a0da39582ef97b91/users/ad0ed8c109cf4c0a965ad26bcdfa4f49/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:30.060273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc19af6-8d39-4775-99aa-64ddb21bc45d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:30.060562 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc19af6-8d39-4775-99aa-64ddb21bc45d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:30.061714 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc19af6-8d39-4775-99aa-64ddb21bc45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:30.077713 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:30.094444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3fc19af6-8d39-4775-99aa-64ddb21bc45d None None] MFA Rules not processed for user `ad0ed8c109cf4c0a965ad26bcdfa4f49`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:30.095691 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3fc19af6-8d39-4775-99aa-64ddb21bc45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:30.168420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1380/2716] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:30.173148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91cd20-de7d-4ae7-9678-073af5359522 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:30.174999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-9a91cd20-de7d-4ae7-9678-073af5359522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:30.241728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a91cd20-de7d-4ae7-9678-073af5359522 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:30.247242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a094d67-b046-46d4-aed6-4354d640c5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:30.247845 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094d67-b046-46d4-aed6-4354d640c5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:30.248121 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094d67-b046-46d4-aed6-4354d640c5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:30.248390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094d67-b046-46d4-aed6-4354d640c5b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:30.249012 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a094d67-b046-46d4-aed6-4354d640c5b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:30.250919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a094d67-b046-46d4-aed6-4354d640c5b3 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:30.293426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc19af6-8d39-4775-99aa-64ddb21bc45d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:30.294875 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3fc19af6-8d39-4775-99aa-64ddb21bc45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:30.312620 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc19af6-8d39-4775-99aa-64ddb21bc45d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:30.315981 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce49170-477c-49eb-a321-797971ee4bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:30.316869 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce49170-477c-49eb-a321-797971ee4bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:30.317271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce49170-477c-49eb-a321-797971ee4bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:30.317600 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce49170-477c-49eb-a321-797971ee4bb9 admin admin] PATH_INFO: `/v3/projects/634e71816d054ebaa641b2f5a81fdad1` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:30.318864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce49170-477c-49eb-a321-797971ee4bb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=634e71816d054ebaa641b2f5a81fdad1)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:30.320910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce49170-477c-49eb-a321-797971ee4bb9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:30.365841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1337/2717] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:30 2022] POST /identity/v3/projects => generated 379 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:30.369785 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a094d67-b046-46d4-aed6-4354d640c5b3 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:30.420686 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dce49170-477c-49eb-a321-797971ee4bb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '634e71816d054ebaa641b2f5a81fdad1', 'request_id': 'req-dce49170-477c-49eb-a321-797971ee4bb9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:30.432985 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a094d67-b046-46d4-aed6-4354d640c5b3 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:30.481877 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:30.488364 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38789dac-52d9-4b75-a546-9c6592d07d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:30.488989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38789dac-52d9-4b75-a546-9c6592d07d57 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:30.489269 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38789dac-52d9-4b75-a546-9c6592d07d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:30.489982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38789dac-52d9-4b75-a546-9c6592d07d57 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:30.490204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38789dac-52d9-4b75-a546-9c6592d07d57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:30.492106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38789dac-52d9-4b75-a546-9c6592d07d57 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:30.512046 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dce49170-477c-49eb-a321-797971ee4bb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '634e71816d054ebaa641b2f5a81fdad1', 'request_id': 'req-dce49170-477c-49eb-a321-797971ee4bb9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:30.518770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dce49170-477c-49eb-a321-797971ee4bb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '634e71816d054ebaa641b2f5a81fdad1', 'request_id': 'req-dce49170-477c-49eb-a321-797971ee4bb9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:30.529637 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dce49170-477c-49eb-a321-797971ee4bb9 admin admin] The token cache is being invalidate because project 634e71816d054ebaa641b2f5a81fdad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:30.529965 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dce49170-477c-49eb-a321-797971ee4bb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:30.530636 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-dce49170-477c-49eb-a321-797971ee4bb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:30.532164 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1381/2718] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:30 2022] DELETE /identity/v3/projects/634e71816d054ebaa641b2f5a81fdad1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:30.536930 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257edcb6-305a-443d-b049-bf06df889363 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:30.537203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257edcb6-305a-443d-b049-bf06df889363 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:30.537470 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257edcb6-305a-443d-b049-bf06df889363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:30.557933 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-257edcb6-305a-443d-b049-bf06df889363 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:30.559146 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-257edcb6-305a-443d-b049-bf06df889363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:30.566970 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1382/2719] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:30.579827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9f3fd-ad56-4055-8d55-c04cf6bbb6be None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:30.580100 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9f3fd-ad56-4055-8d55-c04cf6bbb6be None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:30.580369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a9f3fd-ad56-4055-8d55-c04cf6bbb6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:30.599784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-71a9f3fd-ad56-4055-8d55-c04cf6bbb6be None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:30.600963 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-71a9f3fd-ad56-4055-8d55-c04cf6bbb6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:30.602735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1338/2720] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:30 2022] POST /identity/v3/users => generated 296 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:30.607023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38789dac-52d9-4b75-a546-9c6592d07d57 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:30.608808 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-38789dac-52d9-4b75-a546-9c6592d07d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:30.612570 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1383/2721] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:30.616449 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9f3fd-ad56-4055-8d55-c04cf6bbb6be None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:30.621407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38789dac-52d9-4b75-a546-9c6592d07d57 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:30.626057 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a9f3fd-ad56-4055-8d55-c04cf6bbb6be None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:30.626376 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551661bd-99c9-4eab-86dd-fdf367697625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:30.627003 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551661bd-99c9-4eab-86dd-fdf367697625 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:30.627283 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551661bd-99c9-4eab-86dd-fdf367697625 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:30.627550 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551661bd-99c9-4eab-86dd-fdf367697625 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:30.628207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551661bd-99c9-4eab-86dd-fdf367697625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:30.629842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551661bd-99c9-4eab-86dd-fdf367697625 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:30.630657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3d8f3-b580-48be-aa0f-840e7d8afecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:30.631239 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3d8f3-b580-48be-aa0f-840e7d8afecb admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:30.631514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3d8f3-b580-48be-aa0f-840e7d8afecb admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:30.631784 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3d8f3-b580-48be-aa0f-840e7d8afecb admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:30.632418 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3d8f3-b580-48be-aa0f-840e7d8afecb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:04:30.632717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3d8f3-b580-48be-aa0f-840e7d8afecb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:30.634313 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3d8f3-b580-48be-aa0f-840e7d8afecb admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:30.643843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1339/2722] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:30.645031 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551661bd-99c9-4eab-86dd-fdf367697625 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:30.646189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1384/2723] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:04:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 20:04:30.650824 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3d8f3-b580-48be-aa0f-840e7d8afecb admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:30.655492 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551661bd-99c9-4eab-86dd-fdf367697625 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:30.660253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43001ed9-109f-4e49-9568-7734093edceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:30.660893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43001ed9-109f-4e49-9568-7734093edceb admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:30.661027 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3d8f3-b580-48be-aa0f-840e7d8afecb admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:30.661181 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43001ed9-109f-4e49-9568-7734093edceb admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:30.661441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43001ed9-109f-4e49-9568-7734093edceb admin admin] PATH_INFO: `/v3/projects/b31e2ea5f5604837bab2fc71fdebf548/users/592c0a6ab4b942ceb644c4057f01acd7/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:30.665651 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8640b-2f33-41f7-9232-97cf0c61daaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:30.666914 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8640b-2f33-41f7-9232-97cf0c61daaa admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:30.667191 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8640b-2f33-41f7-9232-97cf0c61daaa admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:30.667457 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8640b-2f33-41f7-9232-97cf0c61daaa admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:30.668063 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8640b-2f33-41f7-9232-97cf0c61daaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:30.669948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8640b-2f33-41f7-9232-97cf0c61daaa admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:30.676788 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43001ed9-109f-4e49-9568-7734093edceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31e2ea5f5604837bab2fc71fdebf548, user_id=592c0a6ab4b942ceb644c4057f01acd7, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:30.679465 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43001ed9-109f-4e49-9568-7734093edceb admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:30.757214 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1340/2724] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:30 2022] PUT /identity/v3/projects/b31e2ea5f5604837bab2fc71fdebf548/users/592c0a6ab4b942ceb644c4057f01acd7/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:30.761047 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43001ed9-109f-4e49-9568-7734093edceb admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:30.782051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:30.789101 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1385/2725] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:30 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:30.793967 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8640b-2f33-41f7-9232-97cf0c61daaa admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:30.820882 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43001ed9-109f-4e49-9568-7734093edceb admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:30.855456 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8640b-2f33-41f7-9232-97cf0c61daaa admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:30.861934 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a6b06-aa78-4b48-a895-28db1d39555f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:30.862472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a6b06-aa78-4b48-a895-28db1d39555f admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:30.862746 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a6b06-aa78-4b48-a895-28db1d39555f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:30.863016 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a6b06-aa78-4b48-a895-28db1d39555f admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:30.863642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a6b06-aa78-4b48-a895-28db1d39555f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:30.865536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a6b06-aa78-4b48-a895-28db1d39555f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:30.880855 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7b577-e594-44ba-9e59-3d9593f4d013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:30.881462 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7b577-e594-44ba-9e59-3d9593f4d013 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:30.881738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7b577-e594-44ba-9e59-3d9593f4d013 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:30.882038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7b577-e594-44ba-9e59-3d9593f4d013 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:30.882670 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7b577-e594-44ba-9e59-3d9593f4d013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:30.884329 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa7b577-e594-44ba-9e59-3d9593f4d013 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:30.894674 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1341/2726] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:30.898673 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7b577-e594-44ba-9e59-3d9593f4d013 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:30.908623 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa7b577-e594-44ba-9e59-3d9593f4d013 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:30.913221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84b93e6-b081-4fe6-a04c-b2fb83212138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:30.913856 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b93e6-b081-4fe6-a04c-b2fb83212138 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:30.914137 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b93e6-b081-4fe6-a04c-b2fb83212138 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:30.914423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84b93e6-b081-4fe6-a04c-b2fb83212138 admin admin] PATH_INFO: `/v3/projects/b31e2ea5f5604837bab2fc71fdebf548/users/592c0a6ab4b942ceb644c4057f01acd7/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:30.916928 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b93e6-b081-4fe6-a04c-b2fb83212138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b31e2ea5f5604837bab2fc71fdebf548, user_id=592c0a6ab4b942ceb644c4057f01acd7, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:30.919648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84b93e6-b081-4fe6-a04c-b2fb83212138 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:30.929173 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1386/2727] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:30 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:30.930413 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1342/2728] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:30 2022] PUT /identity/v3/projects/b31e2ea5f5604837bab2fc71fdebf548/users/592c0a6ab4b942ceb644c4057f01acd7/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:30.933158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a6b06-aa78-4b48-a895-28db1d39555f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:30.942146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8c6da-168f-49b2-b413-1e15169e3363 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:30.942146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8c6da-168f-49b2-b413-1e15169e3363 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:30.942146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8c6da-168f-49b2-b413-1e15169e3363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:30.945179 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a6b06-aa78-4b48-a895-28db1d39555f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:30.970925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-1df8c6da-168f-49b2-b413-1e15169e3363 None None] MFA Rules not processed for user `592c0a6ab4b942ceb644c4057f01acd7`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:30.972146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1df8c6da-168f-49b2-b413-1e15169e3363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:31.002005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:31.002955 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c79a8a-c55b-45e9-8753-66d0aba9a106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:31.003560 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c79a8a-c55b-45e9-8753-66d0aba9a106 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:31.003834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c79a8a-c55b-45e9-8753-66d0aba9a106 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:31.004097 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c79a8a-c55b-45e9-8753-66d0aba9a106 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:31.004722 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c79a8a-c55b-45e9-8753-66d0aba9a106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:31.006334 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c79a8a-c55b-45e9-8753-66d0aba9a106 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:31.019471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1387/2729] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:31.020421 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c79a8a-c55b-45e9-8753-66d0aba9a106 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:31.032750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c79a8a-c55b-45e9-8753-66d0aba9a106 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:31.037608 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0213de0-d756-49fe-9cab-037ae4686fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:31.038266 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0213de0-d756-49fe-9cab-037ae4686fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:31.045950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0213de0-d756-49fe-9cab-037ae4686fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:31.045950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0213de0-d756-49fe-9cab-037ae4686fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:31.045950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0213de0-d756-49fe-9cab-037ae4686fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:31.045950 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0213de0-d756-49fe-9cab-037ae4686fd8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:31.046404 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1343/2730] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:31.054572 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1388/2731] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:31.055666 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8c6da-168f-49b2-b413-1e15169e3363 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:31.056158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0213de0-d756-49fe-9cab-037ae4686fd8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:31.057837 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a0213de0-d756-49fe-9cab-037ae4686fd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:31.066304 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8c6da-168f-49b2-b413-1e15169e3363 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:31.071146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f74add-5e09-47c2-8d1c-97016f4af15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:31.071829 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f74add-5e09-47c2-8d1c-97016f4af15c admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:31.072106 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f74add-5e09-47c2-8d1c-97016f4af15c admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:31.072868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f74add-5e09-47c2-8d1c-97016f4af15c admin admin] PATH_INFO: `/v3/projects/ed1345f3499847dbbf749a99c060271c/users/4b6bbb11f2594a00b39b3cd1ff7e7c59/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:31.083628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:31.088555 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f74add-5e09-47c2-8d1c-97016f4af15c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed1345f3499847dbbf749a99c060271c, user_id=4b6bbb11f2594a00b39b3cd1ff7e7c59, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:31.091301 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f74add-5e09-47c2-8d1c-97016f4af15c admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:31.114618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0213de0-d756-49fe-9cab-037ae4686fd8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:31.118706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ef8bd8-28a7-4a23-8c8c-869cc6adedda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:31.119939 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef8bd8-28a7-4a23-8c8c-869cc6adedda service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:31.120210 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef8bd8-28a7-4a23-8c8c-869cc6adedda service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:31.120474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ef8bd8-28a7-4a23-8c8c-869cc6adedda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:31.129040 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef8bd8-28a7-4a23-8c8c-869cc6adedda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:31.130466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ef8bd8-28a7-4a23-8c8c-869cc6adedda service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:31.139486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1389/2732] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:04:31.204099 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1344/2733] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:31 2022] PUT /identity/v3/projects/ed1345f3499847dbbf749a99c060271c/users/4b6bbb11f2594a00b39b3cd1ff7e7c59/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:31.214928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552003d-5575-4d9e-9a9b-a92d21d3af5a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:31.215207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552003d-5575-4d9e-9a9b-a92d21d3af5a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:31.215474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552003d-5575-4d9e-9a9b-a92d21d3af5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:31.244886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-4552003d-5575-4d9e-9a9b-a92d21d3af5a None None] MFA Rules not processed for user `4b6bbb11f2594a00b39b3cd1ff7e7c59`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:31.246115 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4552003d-5575-4d9e-9a9b-a92d21d3af5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:31.252862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f74add-5e09-47c2-8d1c-97016f4af15c admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:31.254633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-20f74add-5e09-47c2-8d1c-97016f4af15c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:31.311940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1390/2734] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:31.313597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f74add-5e09-47c2-8d1c-97016f4af15c admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:31.316700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552003d-5575-4d9e-9a9b-a92d21d3af5a None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:31.317277 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48ba45-d35c-4514-b4b3-dc5e7d2adc68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:31.317913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48ba45-d35c-4514-b4b3-dc5e7d2adc68 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:31.318195 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48ba45-d35c-4514-b4b3-dc5e7d2adc68 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:31.318467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a48ba45-d35c-4514-b4b3-dc5e7d2adc68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:31.320210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4a48ba45-d35c-4514-b4b3-dc5e7d2adc68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:31.380338 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552003d-5575-4d9e-9a9b-a92d21d3af5a None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:31.381485 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48ba45-d35c-4514-b4b3-dc5e7d2adc68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:31.382971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a48ba45-d35c-4514-b4b3-dc5e7d2adc68 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:31.385183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb3872-0e66-4e30-9715-61e93e83e166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:31.385783 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb3872-0e66-4e30-9715-61e93e83e166 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:31.386092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb3872-0e66-4e30-9715-61e93e83e166 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:31.386371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bb3872-0e66-4e30-9715-61e93e83e166 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:31.386992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb3872-0e66-4e30-9715-61e93e83e166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:31.388919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bb3872-0e66-4e30-9715-61e93e83e166 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:31.391899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1345/2735] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:31.491323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1391/2736] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:31 2022] POST /identity/v3/projects => generated 383 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:31.495490 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48ba45-d35c-4514-b4b3-dc5e7d2adc68 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:31.506479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a48ba45-d35c-4514-b4b3-dc5e7d2adc68 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:31.565678 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:31.590242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8349635-55c8-4b55-8161-5c2be33c84d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:31.590242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8349635-55c8-4b55-8161-5c2be33c84d3 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:31.590242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8349635-55c8-4b55-8161-5c2be33c84d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:31.590242 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8349635-55c8-4b55-8161-5c2be33c84d3 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:31.590610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8349635-55c8-4b55-8161-5c2be33c84d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:31.590610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8349635-55c8-4b55-8161-5c2be33c84d3 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:31.693209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1346/2737] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:31 2022] POST /identity/v3/users => generated 292 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:31.697238 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb3872-0e66-4e30-9715-61e93e83e166 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:31.707290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bb3872-0e66-4e30-9715-61e93e83e166 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:31.711947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bf7a0-04dd-4754-a126-4ceb971eae7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:31.713292 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bf7a0-04dd-4754-a126-4ceb971eae7f admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:31.713570 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bf7a0-04dd-4754-a126-4ceb971eae7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:31.713858 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5bf7a0-04dd-4754-a126-4ceb971eae7f admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:31.714514 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bf7a0-04dd-4754-a126-4ceb971eae7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:31.716072 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5bf7a0-04dd-4754-a126-4ceb971eae7f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:31.726028 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1392/2738] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:31.730584 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8349635-55c8-4b55-8161-5c2be33c84d3 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:31.740462 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8349635-55c8-4b55-8161-5c2be33c84d3 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:31.745071 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96097e8d-8ab4-4b3e-9647-4bdcdcc14a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:31.745663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96097e8d-8ab4-4b3e-9647-4bdcdcc14a82 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:31.745999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96097e8d-8ab4-4b3e-9647-4bdcdcc14a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:31.746274 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96097e8d-8ab4-4b3e-9647-4bdcdcc14a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:31.746906 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96097e8d-8ab4-4b3e-9647-4bdcdcc14a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:31.748486 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96097e8d-8ab4-4b3e-9647-4bdcdcc14a82 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:31.758353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1347/2739] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:31.763160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bf7a0-04dd-4754-a126-4ceb971eae7f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:31.773273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5bf7a0-04dd-4754-a126-4ceb971eae7f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:31.778913 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a32c60-7b58-44f4-a357-eacd07ea6332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:31.779557 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a32c60-7b58-44f4-a357-eacd07ea6332 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:31.779834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a32c60-7b58-44f4-a357-eacd07ea6332 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:31.780105 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a32c60-7b58-44f4-a357-eacd07ea6332 admin admin] PATH_INFO: `/v3/projects/6b2c96cf9cba4894aff5be68c377c74c/users/3e6655f9abd74f7c8ce4e894e738b7a1/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:31.794701 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a32c60-7b58-44f4-a357-eacd07ea6332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b2c96cf9cba4894aff5be68c377c74c, user_id=3e6655f9abd74f7c8ce4e894e738b7a1, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:31.798500 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a32c60-7b58-44f4-a357-eacd07ea6332 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:31.880163 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1393/2740] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:31 2022] PUT /identity/v3/projects/6b2c96cf9cba4894aff5be68c377c74c/users/3e6655f9abd74f7c8ce4e894e738b7a1/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:31.891872 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f9dab-481b-4fea-83f3-975d7ef48cef None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:31.892154 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f9dab-481b-4fea-83f3-975d7ef48cef None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:31.892424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031f9dab-481b-4fea-83f3-975d7ef48cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:31.920737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-031f9dab-481b-4fea-83f3-975d7ef48cef None None] MFA Rules not processed for user `3e6655f9abd74f7c8ce4e894e738b7a1`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:31.922031 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-031f9dab-481b-4fea-83f3-975d7ef48cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:31.986792 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1348/2741] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:31.991534 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a32c60-7b58-44f4-a357-eacd07ea6332 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:31.997215 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:32.051536 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a32c60-7b58-44f4-a357-eacd07ea6332 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:32.056337 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d056da-a9f9-44f8-8b14-5de24da6b527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:32.056936 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d056da-a9f9-44f8-8b14-5de24da6b527 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:32.057208 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d056da-a9f9-44f8-8b14-5de24da6b527 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:32.057477 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d056da-a9f9-44f8-8b14-5de24da6b527 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:32.058118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d056da-a9f9-44f8-8b14-5de24da6b527 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:32.059998 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d056da-a9f9-44f8-8b14-5de24da6b527 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:32.199853 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1394/2742] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:31 2022] POST /identity/v3/projects => generated 383 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:32.203900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f9dab-481b-4fea-83f3-975d7ef48cef None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:32.214997 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031f9dab-481b-4fea-83f3-975d7ef48cef None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:32.274168 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fc446-3d1a-4d99-82b5-b410345ef986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:32.274900 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fc446-3d1a-4d99-82b5-b410345ef986 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:32.275178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fc446-3d1a-4d99-82b5-b410345ef986 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:32.275472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604fc446-3d1a-4d99-82b5-b410345ef986 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:32.276113 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fc446-3d1a-4d99-82b5-b410345ef986 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:32.278068 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604fc446-3d1a-4d99-82b5-b410345ef986 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:32.366820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1349/2743] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:32 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:32.370587 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d056da-a9f9-44f8-8b14-5de24da6b527 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:32.380147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d056da-a9f9-44f8-8b14-5de24da6b527 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:32.384728 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d159a-76bf-43aa-8be5-2148a41b70a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:32.385925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d159a-76bf-43aa-8be5-2148a41b70a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:32.386201 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d159a-76bf-43aa-8be5-2148a41b70a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:32.386464 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640d159a-76bf-43aa-8be5-2148a41b70a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:32.387098 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d159a-76bf-43aa-8be5-2148a41b70a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:32.388756 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640d159a-76bf-43aa-8be5-2148a41b70a3 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:32.399255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1395/2744] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:32.403587 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fc446-3d1a-4d99-82b5-b410345ef986 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:32.413782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604fc446-3d1a-4d99-82b5-b410345ef986 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:32.418574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad39103-8670-4411-8569-57da707ee5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:32.419210 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad39103-8670-4411-8569-57da707ee5d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:32.419488 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad39103-8670-4411-8569-57da707ee5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:32.419759 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad39103-8670-4411-8569-57da707ee5d1 admin admin] PATH_INFO: `/v3/projects/3438f0c510cf473a93c2bd0e95f669d6/users/7c5256688a7845809b227de554e09ae6/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:32.435027 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad39103-8670-4411-8569-57da707ee5d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3438f0c510cf473a93c2bd0e95f669d6, user_id=7c5256688a7845809b227de554e09ae6, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:32.438079 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad39103-8670-4411-8569-57da707ee5d1 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:32.562412 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1350/2745] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:32 2022] PUT /identity/v3/projects/3438f0c510cf473a93c2bd0e95f669d6/users/7c5256688a7845809b227de554e09ae6/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:32.566783 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d159a-76bf-43aa-8be5-2148a41b70a3 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:32.576968 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640d159a-76bf-43aa-8be5-2148a41b70a3 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:32.639654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d3850-ea6d-4411-8460-4b25b1db575d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:32.640287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d3850-ea6d-4411-8460-4b25b1db575d admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:32.640568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d3850-ea6d-4411-8460-4b25b1db575d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:32.640835 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d3850-ea6d-4411-8460-4b25b1db575d admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:32.641460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d3850-ea6d-4411-8460-4b25b1db575d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:32.643062 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d3850-ea6d-4411-8460-4b25b1db575d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:32.653308 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1396/2746] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:32.657355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad39103-8670-4411-8569-57da707ee5d1 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:32.668260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad39103-8670-4411-8569-57da707ee5d1 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:32.673762 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33808c-15f5-4ea1-9eaf-f38e9259ed3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:32.674508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33808c-15f5-4ea1-9eaf-f38e9259ed3a admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:32.674852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33808c-15f5-4ea1-9eaf-f38e9259ed3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:32.675186 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33808c-15f5-4ea1-9eaf-f38e9259ed3a admin admin] PATH_INFO: `/v3/projects/3438f0c510cf473a93c2bd0e95f669d6/users/7c5256688a7845809b227de554e09ae6/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:32.677158 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33808c-15f5-4ea1-9eaf-f38e9259ed3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3438f0c510cf473a93c2bd0e95f669d6, user_id=7c5256688a7845809b227de554e09ae6, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:32.679894 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33808c-15f5-4ea1-9eaf-f38e9259ed3a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:32.691913 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1351/2747] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:32 2022] PUT /identity/v3/projects/3438f0c510cf473a93c2bd0e95f669d6/users/7c5256688a7845809b227de554e09ae6/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:32.701909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3543a-2dd0-492f-9049-3975038332d4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:32.702387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3543a-2dd0-492f-9049-3975038332d4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:32.702519 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c3543a-2dd0-492f-9049-3975038332d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:32.730901 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-47c3543a-2dd0-492f-9049-3975038332d4 None None] MFA Rules not processed for user `7c5256688a7845809b227de554e09ae6`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:32.732120 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-47c3543a-2dd0-492f-9049-3975038332d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:32.803919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:32.808059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1397/2748] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:32.819742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33808c-15f5-4ea1-9eaf-f38e9259ed3a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:32.838095 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:32.883025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33808c-15f5-4ea1-9eaf-f38e9259ed3a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:32.888618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ac145-e38a-45eb-8214-3338d3201130 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:32.889648 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ac145-e38a-45eb-8214-3338d3201130 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:32.894608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ac145-e38a-45eb-8214-3338d3201130 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:32.894608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ac145-e38a-45eb-8214-3338d3201130 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:32.962345 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ac145-e38a-45eb-8214-3338d3201130 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:32.964317 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ac145-e38a-45eb-8214-3338d3201130 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:32.973456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1352/2749] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:35.097967 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3543a-2dd0-492f-9049-3975038332d4 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:35.108011 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c3543a-2dd0-492f-9049-3975038332d4 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:35.111618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1de2d-b7c7-4448-8dfa-4510501b419f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:35.112213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1de2d-b7c7-4448-8dfa-4510501b419f service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:35.112490 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1de2d-b7c7-4448-8dfa-4510501b419f service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:35.112760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1de2d-b7c7-4448-8dfa-4510501b419f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:35.168220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1de2d-b7c7-4448-8dfa-4510501b419f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:35.169624 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1de2d-b7c7-4448-8dfa-4510501b419f service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:35.178203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1398/2750] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:35.698589 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ac145-e38a-45eb-8214-3338d3201130 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:35.760820 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ac145-e38a-45eb-8214-3338d3201130 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:35.765558 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce5129-2115-4a4e-917c-6ae3a6f56ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:35.766209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce5129-2115-4a4e-917c-6ae3a6f56ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:35.766485 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce5129-2115-4a4e-917c-6ae3a6f56ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:35.766753 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce5129-2115-4a4e-917c-6ae3a6f56ff8 admin admin] PATH_INFO: `/v3/users/b19ac61d48034c0cae49e77759d646e5` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:35.767906 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce5129-2115-4a4e-917c-6ae3a6f56ff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b19ac61d48034c0cae49e77759d646e5)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:35.769824 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce5129-2115-4a4e-917c-6ae3a6f56ff8 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:35.931446 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-17ce5129-2115-4a4e-917c-6ae3a6f56ff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b19ac61d48034c0cae49e77759d646e5'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:35.944493 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-17ce5129-2115-4a4e-917c-6ae3a6f56ff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b19ac61d48034c0cae49e77759d646e5'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:36.014738 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-17ce5129-2115-4a4e-917c-6ae3a6f56ff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b19ac61d48034c0cae49e77759d646e5'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:36.015354 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-17ce5129-2115-4a4e-917c-6ae3a6f56ff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b19ac61d48034c0cae49e77759d646e5'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:36.016657 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:36.023080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-17ce5129-2115-4a4e-917c-6ae3a6f56ff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b19ac61d48034c0cae49e77759d646e5'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:36.028973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1353/2751] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:35 2022] DELETE /identity/v3/users/b19ac61d48034c0cae49e77759d646e5 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:36.033521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91348e-3ade-4a16-a52c-e85b620a873d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:36.033910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91348e-3ade-4a16-a52c-e85b620a873d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:36.034255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91348e-3ade-4a16-a52c-e85b620a873d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:36.052871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2f91348e-3ade-4a16-a52c-e85b620a873d None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:36.054138 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2f91348e-3ade-4a16-a52c-e85b620a873d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:36.113913 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1399/2752] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:36.124361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce5129-2115-4a4e-917c-6ae3a6f56ff8 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:36.125825 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-17ce5129-2115-4a4e-917c-6ae3a6f56ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:36.183474 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce5129-2115-4a4e-917c-6ae3a6f56ff8 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:36.188265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658f5b9-e9c5-4f0c-a6df-702d986ab5f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:36.188874 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658f5b9-e9c5-4f0c-a6df-702d986ab5f0 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:36.189154 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658f5b9-e9c5-4f0c-a6df-702d986ab5f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:36.189431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658f5b9-e9c5-4f0c-a6df-702d986ab5f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:36.199265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658f5b9-e9c5-4f0c-a6df-702d986ab5f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:36.200688 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658f5b9-e9c5-4f0c-a6df-702d986ab5f0 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:36.210171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1354/2753] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:36.729871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91348e-3ade-4a16-a52c-e85b620a873d None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:36.731260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2f91348e-3ade-4a16-a52c-e85b620a873d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:36.788727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91348e-3ade-4a16-a52c-e85b620a873d None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:36.792818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4de7c-1da5-4b6a-a582-8b6cda2f1632 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:36.793415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4de7c-1da5-4b6a-a582-8b6cda2f1632 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:36.793693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4de7c-1da5-4b6a-a582-8b6cda2f1632 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:36.794059 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4de7c-1da5-4b6a-a582-8b6cda2f1632 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:36.795759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-36d4de7c-1da5-4b6a-a582-8b6cda2f1632 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:36.844847 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=100 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:36.857220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4de7c-1da5-4b6a-a582-8b6cda2f1632 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:36.858749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4de7c-1da5-4b6a-a582-8b6cda2f1632 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:36.868220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1400/2754] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:36 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:36.902835 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658f5b9-e9c5-4f0c-a6df-702d986ab5f0 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:36.904330 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c658f5b9-e9c5-4f0c-a6df-702d986ab5f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:36.918768 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658f5b9-e9c5-4f0c-a6df-702d986ab5f0 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:36.923602 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:36.924266 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:36.924546 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:36.924821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] PATH_INFO: `/v3/projects/291c212967264fbfbb6dd51b92eb5220` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:36.926022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=291c212967264fbfbb6dd51b92eb5220)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:36.927929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:37.038537 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '291c212967264fbfbb6dd51b92eb5220', 'request_id': 'req-c65bf049-0683-482f-8574-3cc7b120b6d1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:37.044104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '291c212967264fbfbb6dd51b92eb5220', 'request_id': 'req-c65bf049-0683-482f-8574-3cc7b120b6d1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:37.051366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '291c212967264fbfbb6dd51b92eb5220', 'request_id': 'req-c65bf049-0683-482f-8574-3cc7b120b6d1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:37.117031 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] The token cache is being invalidate because project 291c212967264fbfbb6dd51b92eb5220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:37.117362 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:37.117977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:37.119456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1355/2755] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:36 2022] DELETE /identity/v3/projects/291c212967264fbfbb6dd51b92eb5220 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:37.123474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4de7c-1da5-4b6a-a582-8b6cda2f1632 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:37.133660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4de7c-1da5-4b6a-a582-8b6cda2f1632 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:37.190091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e01cc-9d59-4a08-8817-c1d355eb75c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:37.190637 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e01cc-9d59-4a08-8817-c1d355eb75c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:37.190909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e01cc-9d59-4a08-8817-c1d355eb75c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:37.191176 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399e01cc-9d59-4a08-8817-c1d355eb75c3 admin admin] PATH_INFO: `/v3/users/ad0ed8c109cf4c0a965ad26bcdfa4f49` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:37.192255 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e01cc-9d59-4a08-8817-c1d355eb75c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad0ed8c109cf4c0a965ad26bcdfa4f49)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:37.194222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399e01cc-9d59-4a08-8817-c1d355eb75c3 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:37.392373 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-399e01cc-9d59-4a08-8817-c1d355eb75c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad0ed8c109cf4c0a965ad26bcdfa4f49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:37.483681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-399e01cc-9d59-4a08-8817-c1d355eb75c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad0ed8c109cf4c0a965ad26bcdfa4f49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:37.496618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-399e01cc-9d59-4a08-8817-c1d355eb75c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad0ed8c109cf4c0a965ad26bcdfa4f49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:37.497233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-399e01cc-9d59-4a08-8817-c1d355eb75c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad0ed8c109cf4c0a965ad26bcdfa4f49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:37.497924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-399e01cc-9d59-4a08-8817-c1d355eb75c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad0ed8c109cf4c0a965ad26bcdfa4f49'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:37.510531 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1401/2756] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:37 2022] DELETE /identity/v3/users/ad0ed8c109cf4c0a965ad26bcdfa4f49 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:37.993472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:37.997955 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:38.008798 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bf049-0683-482f-8574-3cc7b120b6d1 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:38.065704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:38.066351 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:38.066640 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:38.066888 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] PATH_INFO: `/v3/projects/32dd354712424871a0da39582ef97b91` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:38.067977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32dd354712424871a0da39582ef97b91)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:38.069888 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:38.086454 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:38.187275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32dd354712424871a0da39582ef97b91', 'request_id': 'req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:38.192636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32dd354712424871a0da39582ef97b91', 'request_id': 'req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:38.199409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32dd354712424871a0da39582ef97b91', 'request_id': 'req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:38.275360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] The token cache is being invalidate because project 32dd354712424871a0da39582ef97b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:38.275553 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:38.276157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:38.277774 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1356/2757] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:37 2022] DELETE /identity/v3/projects/32dd354712424871a0da39582ef97b91 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:38.281773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e01cc-9d59-4a08-8817-c1d355eb75c3 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:38.284231 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-399e01cc-9d59-4a08-8817-c1d355eb75c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:38.296891 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399e01cc-9d59-4a08-8817-c1d355eb75c3 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:38.351507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5e231-01dd-40ad-bbc2-3fcd9c8eb94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:38.352119 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5e231-01dd-40ad-bbc2-3fcd9c8eb94d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:38.352394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5e231-01dd-40ad-bbc2-3fcd9c8eb94d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:38.352658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5e231-01dd-40ad-bbc2-3fcd9c8eb94d admin admin] PATH_INFO: `/v3/users/592c0a6ab4b942ceb644c4057f01acd7` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:38.353770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5e231-01dd-40ad-bbc2-3fcd9c8eb94d admin admin] RBAC: Authorizing `identity:delete_user(user_id=592c0a6ab4b942ceb644c4057f01acd7)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:38.355720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5e231-01dd-40ad-bbc2-3fcd9c8eb94d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:38.365404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:38.600433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-53c5e231-01dd-40ad-bbc2-3fcd9c8eb94d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '592c0a6ab4b942ceb644c4057f01acd7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:38.677988 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-53c5e231-01dd-40ad-bbc2-3fcd9c8eb94d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '592c0a6ab4b942ceb644c4057f01acd7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:38.691729 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-53c5e231-01dd-40ad-bbc2-3fcd9c8eb94d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '592c0a6ab4b942ceb644c4057f01acd7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:38.692382 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-53c5e231-01dd-40ad-bbc2-3fcd9c8eb94d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '592c0a6ab4b942ceb644c4057f01acd7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:38.692994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-53c5e231-01dd-40ad-bbc2-3fcd9c8eb94d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '592c0a6ab4b942ceb644c4057f01acd7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:38.705278 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1402/2758] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:38 2022] DELETE /identity/v3/users/592c0a6ab4b942ceb644c4057f01acd7 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:39.148005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:39.149533 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:39.162852 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9698c-84cd-4ab4-b998-c0cf88d2de98 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:39.219848 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73af129-d652-44e5-b009-9cbb20768bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:39.220481 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73af129-d652-44e5-b009-9cbb20768bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:39.220761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73af129-d652-44e5-b009-9cbb20768bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:39.221031 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73af129-d652-44e5-b009-9cbb20768bd1 admin admin] PATH_INFO: `/v3/projects/b31e2ea5f5604837bab2fc71fdebf548` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:39.222295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73af129-d652-44e5-b009-9cbb20768bd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b31e2ea5f5604837bab2fc71fdebf548)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:39.224204 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73af129-d652-44e5-b009-9cbb20768bd1 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:39.323264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-b73af129-d652-44e5-b009-9cbb20768bd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b31e2ea5f5604837bab2fc71fdebf548', 'request_id': 'req-b73af129-d652-44e5-b009-9cbb20768bd1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:39.328728 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-b73af129-d652-44e5-b009-9cbb20768bd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b31e2ea5f5604837bab2fc71fdebf548', 'request_id': 'req-b73af129-d652-44e5-b009-9cbb20768bd1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:39.334847 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-b73af129-d652-44e5-b009-9cbb20768bd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b31e2ea5f5604837bab2fc71fdebf548', 'request_id': 'req-b73af129-d652-44e5-b009-9cbb20768bd1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:39.403661 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-b73af129-d652-44e5-b009-9cbb20768bd1 admin admin] The token cache is being invalidate because project b31e2ea5f5604837bab2fc71fdebf548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:39.403891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-b73af129-d652-44e5-b009-9cbb20768bd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:39.404511 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-b73af129-d652-44e5-b009-9cbb20768bd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:39.406038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1357/2759] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:39 2022] DELETE /identity/v3/projects/b31e2ea5f5604837bab2fc71fdebf548 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:39.411829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a96976-2753-4990-9709-a6aa5cc94913 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:39.412287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a96976-2753-4990-9709-a6aa5cc94913 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:39.412420 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a96976-2753-4990-9709-a6aa5cc94913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:39.431618 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c9a96976-2753-4990-9709-a6aa5cc94913 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:39.432798 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c9a96976-2753-4990-9709-a6aa5cc94913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:39.490436 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1403/2760] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:39.552999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7a210-7872-47d8-a84b-fdb8e25d124c None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:39.553278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7a210-7872-47d8-a84b-fdb8e25d124c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:39.553548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7a210-7872-47d8-a84b-fdb8e25d124c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:39.573980 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-2db7a210-7872-47d8-a84b-fdb8e25d124c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:39.575216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2db7a210-7872-47d8-a84b-fdb8e25d124c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:39.582981 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1358/2761] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:39.587118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a96976-2753-4990-9709-a6aa5cc94913 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:39.597592 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a96976-2753-4990-9709-a6aa5cc94913 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:39.602197 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a7a6b-b9f1-408e-9b6b-feb472fedd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:39.602789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a7a6b-b9f1-408e-9b6b-feb472fedd95 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:39.603065 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a7a6b-b9f1-408e-9b6b-feb472fedd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:39.603333 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687a7a6b-b9f1-408e-9b6b-feb472fedd95 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:39.603996 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a7a6b-b9f1-408e-9b6b-feb472fedd95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:04:39.604320 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a7a6b-b9f1-408e-9b6b-feb472fedd95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:39.605912 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687a7a6b-b9f1-408e-9b6b-feb472fedd95 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:39.617207 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1404/2762] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:04:39.622086 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7a210-7872-47d8-a84b-fdb8e25d124c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:39.631252 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7a210-7872-47d8-a84b-fdb8e25d124c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:39.636563 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7977c33-0fb9-4ed8-a03e-29a5309df238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:39.638241 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7977c33-0fb9-4ed8-a03e-29a5309df238 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:39.638241 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7977c33-0fb9-4ed8-a03e-29a5309df238 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:39.638241 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7977c33-0fb9-4ed8-a03e-29a5309df238 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:39.638505 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7977c33-0fb9-4ed8-a03e-29a5309df238 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:39.640286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7977c33-0fb9-4ed8-a03e-29a5309df238 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:39.724916 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1359/2763] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:39 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:39.729007 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a7a6b-b9f1-408e-9b6b-feb472fedd95 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:39.738972 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687a7a6b-b9f1-408e-9b6b-feb472fedd95 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:39.794131 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab68a3-c5ef-4262-a22f-6fb407a3b783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:39.794726 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab68a3-c5ef-4262-a22f-6fb407a3b783 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:39.795003 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab68a3-c5ef-4262-a22f-6fb407a3b783 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:39.795274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab68a3-c5ef-4262-a22f-6fb407a3b783 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:39.795894 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab68a3-c5ef-4262-a22f-6fb407a3b783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:39.797789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab68a3-c5ef-4262-a22f-6fb407a3b783 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:39.953009 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1405/2764] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:39 2022] POST /identity/v3/users => generated 294 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:39.959569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7977c33-0fb9-4ed8-a03e-29a5309df238 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:39.970005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7977c33-0fb9-4ed8-a03e-29a5309df238 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:39.974528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d399f0e-3cdf-4d44-ae93-650cb3432977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:39.975146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d399f0e-3cdf-4d44-ae93-650cb3432977 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:39.975371 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d399f0e-3cdf-4d44-ae93-650cb3432977 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:39.975640 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d399f0e-3cdf-4d44-ae93-650cb3432977 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:39.976264 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d399f0e-3cdf-4d44-ae93-650cb3432977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:39.977866 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d399f0e-3cdf-4d44-ae93-650cb3432977 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:39.979636 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:39.988520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1360/2765] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:39.992438 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab68a3-c5ef-4262-a22f-6fb407a3b783 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:39.999395 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:40.003957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab68a3-c5ef-4262-a22f-6fb407a3b783 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:40.008516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df493c0a-5308-4c29-b9fc-5c4c55c889a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:40.009104 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df493c0a-5308-4c29-b9fc-5c4c55c889a9 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:40.009379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df493c0a-5308-4c29-b9fc-5c4c55c889a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:40.009648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df493c0a-5308-4c29-b9fc-5c4c55c889a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:40.010306 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df493c0a-5308-4c29-b9fc-5c4c55c889a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:40.011870 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df493c0a-5308-4c29-b9fc-5c4c55c889a9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:40.022389 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1406/2766] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:40.026634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d399f0e-3cdf-4d44-ae93-650cb3432977 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:40.036659 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d399f0e-3cdf-4d44-ae93-650cb3432977 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:40.047015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63873ea2-d2fd-472c-bcd8-9a3da718bc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:40.047015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63873ea2-d2fd-472c-bcd8-9a3da718bc52 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:40.047015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63873ea2-d2fd-472c-bcd8-9a3da718bc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:40.047015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63873ea2-d2fd-472c-bcd8-9a3da718bc52 admin admin] PATH_INFO: `/v3/projects/395ace0757fa4822bdf760af3248a0b9/users/7f2b8b7c42a243cfb105ef84d6bf9ed0/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:40.061118 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63873ea2-d2fd-472c-bcd8-9a3da718bc52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=395ace0757fa4822bdf760af3248a0b9, user_id=7f2b8b7c42a243cfb105ef84d6bf9ed0, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:40.063804 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63873ea2-d2fd-472c-bcd8-9a3da718bc52 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:40.124409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1361/2767] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:40 2022] PUT /identity/v3/projects/395ace0757fa4822bdf760af3248a0b9/users/7f2b8b7c42a243cfb105ef84d6bf9ed0/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:40.145486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871ff43-58ca-4e38-b459-36555aa85392 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:40.145772 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871ff43-58ca-4e38-b459-36555aa85392 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:40.146084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871ff43-58ca-4e38-b459-36555aa85392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:40.174516 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-3871ff43-58ca-4e38-b459-36555aa85392 None None] MFA Rules not processed for user `7f2b8b7c42a243cfb105ef84d6bf9ed0`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:40.175744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-3871ff43-58ca-4e38-b459-36555aa85392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:40.238488 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1407/2768] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:40.242936 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63873ea2-d2fd-472c-bcd8-9a3da718bc52 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:40.307750 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63873ea2-d2fd-472c-bcd8-9a3da718bc52 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:40.312775 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665c73f-6a52-4d32-8359-93bbf53eadec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:40.313374 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665c73f-6a52-4d32-8359-93bbf53eadec admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:40.313649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665c73f-6a52-4d32-8359-93bbf53eadec admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:40.313944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a665c73f-6a52-4d32-8359-93bbf53eadec admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:40.328035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665c73f-6a52-4d32-8359-93bbf53eadec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:40.328035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a665c73f-6a52-4d32-8359-93bbf53eadec admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:40.410611 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1362/2769] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:40 2022] POST /identity/v3/projects => generated 387 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:40.414655 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871ff43-58ca-4e38-b459-36555aa85392 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:40.425999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871ff43-58ca-4e38-b459-36555aa85392 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:40.482774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6cef5d-2d8d-4e49-a309-1fcdf3d2998a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:40.483377 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6cef5d-2d8d-4e49-a309-1fcdf3d2998a admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:40.483654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6cef5d-2d8d-4e49-a309-1fcdf3d2998a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:40.483925 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6cef5d-2d8d-4e49-a309-1fcdf3d2998a admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:40.484546 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6cef5d-2d8d-4e49-a309-1fcdf3d2998a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:40.486486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6cef5d-2d8d-4e49-a309-1fcdf3d2998a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:40.649319 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1408/2770] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:40 2022] POST /identity/v3/users => generated 300 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:40.653739 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665c73f-6a52-4d32-8359-93bbf53eadec admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:40.663844 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a665c73f-6a52-4d32-8359-93bbf53eadec admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:40.669038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9ad56-36f9-4e2b-9870-7b288dcb5560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:40.669624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ad56-36f9-4e2b-9870-7b288dcb5560 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:40.669927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ad56-36f9-4e2b-9870-7b288dcb5560 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:40.670200 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ad56-36f9-4e2b-9870-7b288dcb5560 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:40.670849 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9ad56-36f9-4e2b-9870-7b288dcb5560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:40.672424 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9ad56-36f9-4e2b-9870-7b288dcb5560 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:40.682786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1363/2771] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:40.687337 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6cef5d-2d8d-4e49-a309-1fcdf3d2998a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:40.699866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6cef5d-2d8d-4e49-a309-1fcdf3d2998a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:40.704669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef1104-38b2-44d3-a302-bbccca10e772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:40.707085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef1104-38b2-44d3-a302-bbccca10e772 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:40.707085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef1104-38b2-44d3-a302-bbccca10e772 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:40.707085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef1104-38b2-44d3-a302-bbccca10e772 admin admin] PATH_INFO: `/v3/projects/67798287cbfd4bcc951c03ab802782bc/users/5dffce967c3f43b9833effa5e7874530/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:40.723162 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef1104-38b2-44d3-a302-bbccca10e772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67798287cbfd4bcc951c03ab802782bc, user_id=5dffce967c3f43b9833effa5e7874530, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:40.725913 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ef1104-38b2-44d3-a302-bbccca10e772 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:40.844681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1409/2772] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:40 2022] PUT /identity/v3/projects/67798287cbfd4bcc951c03ab802782bc/users/5dffce967c3f43b9833effa5e7874530/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:40.848647 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9ad56-36f9-4e2b-9870-7b288dcb5560 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:40.858679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9ad56-36f9-4e2b-9870-7b288dcb5560 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:40.915193 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca254be-2525-4d55-9820-f77f3ba51db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:40.915780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca254be-2525-4d55-9820-f77f3ba51db9 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:40.916057 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca254be-2525-4d55-9820-f77f3ba51db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:40.916328 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca254be-2525-4d55-9820-f77f3ba51db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:40.916949 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca254be-2525-4d55-9820-f77f3ba51db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:40.918600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca254be-2525-4d55-9820-f77f3ba51db9 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:40.928387 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1364/2773] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:40.932556 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef1104-38b2-44d3-a302-bbccca10e772 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:40.943316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef1104-38b2-44d3-a302-bbccca10e772 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:40.947650 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1053498-c084-45ae-a985-220ec51f337a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:40.948299 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1053498-c084-45ae-a985-220ec51f337a admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:40.948578 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1053498-c084-45ae-a985-220ec51f337a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:40.948850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1053498-c084-45ae-a985-220ec51f337a admin admin] PATH_INFO: `/v3/projects/67798287cbfd4bcc951c03ab802782bc/users/5dffce967c3f43b9833effa5e7874530/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:40.951289 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1053498-c084-45ae-a985-220ec51f337a admin admin] RBAC: Authorizing `identity:create_grant(project_id=67798287cbfd4bcc951c03ab802782bc, user_id=5dffce967c3f43b9833effa5e7874530, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:40.953947 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1053498-c084-45ae-a985-220ec51f337a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:40.964365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1410/2774] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:40 2022] PUT /identity/v3/projects/67798287cbfd4bcc951c03ab802782bc/users/5dffce967c3f43b9833effa5e7874530/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:40.976292 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142867f-7926-4b88-a867-b7668168433f None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:40.976601 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142867f-7926-4b88-a867-b7668168433f None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:40.976876 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0142867f-7926-4b88-a867-b7668168433f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:41.004037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-0142867f-7926-4b88-a867-b7668168433f None None] MFA Rules not processed for user `5dffce967c3f43b9833effa5e7874530`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:41.005243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0142867f-7926-4b88-a867-b7668168433f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:41.035615 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:41.072355 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1365/2775] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:41.084109 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1053498-c084-45ae-a985-220ec51f337a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:41.085931 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-b1053498-c084-45ae-a985-220ec51f337a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:41.127460 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:41.145381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1053498-c084-45ae-a985-220ec51f337a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:41.149140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2531f894-d6a6-45a3-b9a2-7f4010702a65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:41.149732 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531f894-d6a6-45a3-b9a2-7f4010702a65 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:41.150047 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531f894-d6a6-45a3-b9a2-7f4010702a65 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:41.150320 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2531f894-d6a6-45a3-b9a2-7f4010702a65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:41.159078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531f894-d6a6-45a3-b9a2-7f4010702a65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:41.160494 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2531f894-d6a6-45a3-b9a2-7f4010702a65 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:41.169726 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1411/2776] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:41.819172 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142867f-7926-4b88-a867-b7668168433f None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:41.820794 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0142867f-7926-4b88-a867-b7668168433f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:41.887582 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0142867f-7926-4b88-a867-b7668168433f None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:41.892324 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31316d-f999-4c15-8c04-85076e40f9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:41.892954 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31316d-f999-4c15-8c04-85076e40f9a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:41.893235 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31316d-f999-4c15-8c04-85076e40f9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:41.893509 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31316d-f999-4c15-8c04-85076e40f9a9 admin admin] PATH_INFO: `/v3/users/4b6bbb11f2594a00b39b3cd1ff7e7c59` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:41.894705 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31316d-f999-4c15-8c04-85076e40f9a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b6bbb11f2594a00b39b3cd1ff7e7c59)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:41.896631 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31316d-f999-4c15-8c04-85076e40f9a9 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:42.177888 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-cf31316d-f999-4c15-8c04-85076e40f9a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b6bbb11f2594a00b39b3cd1ff7e7c59'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:42.189878 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-cf31316d-f999-4c15-8c04-85076e40f9a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b6bbb11f2594a00b39b3cd1ff7e7c59'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:42.317973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-cf31316d-f999-4c15-8c04-85076e40f9a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b6bbb11f2594a00b39b3cd1ff7e7c59'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:42.318724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-cf31316d-f999-4c15-8c04-85076e40f9a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b6bbb11f2594a00b39b3cd1ff7e7c59'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:42.326684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-cf31316d-f999-4c15-8c04-85076e40f9a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b6bbb11f2594a00b39b3cd1ff7e7c59'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:42.332507 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1366/2777] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:41 2022] DELETE /identity/v3/users/4b6bbb11f2594a00b39b3cd1ff7e7c59 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:42.336948 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082482-1ea0-4839-bb85-aa2b11667d91 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:42.337714 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082482-1ea0-4839-bb85-aa2b11667d91 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:42.337714 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082482-1ea0-4839-bb85-aa2b11667d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:42.358189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-ca082482-1ea0-4839-bb85-aa2b11667d91 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:42.359513 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ca082482-1ea0-4839-bb85-aa2b11667d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:42.425321 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1412/2778] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:42.435724 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31316d-f999-4c15-8c04-85076e40f9a9 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:42.437149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-cf31316d-f999-4c15-8c04-85076e40f9a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:42.489288 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:42.494469 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31316d-f999-4c15-8c04-85076e40f9a9 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:42.498096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d726d5-be18-4f03-95df-244953ec8e13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:42.498690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d726d5-be18-4f03-95df-244953ec8e13 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:42.498965 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d726d5-be18-4f03-95df-244953ec8e13 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:42.499233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d726d5-be18-4f03-95df-244953ec8e13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:42.507903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d726d5-be18-4f03-95df-244953ec8e13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:42.509315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d726d5-be18-4f03-95df-244953ec8e13 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:42.519035 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1367/2779] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:42.561834 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca082482-1ea0-4839-bb85-aa2b11667d91 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:42.563311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-ca082482-1ea0-4839-bb85-aa2b11667d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:42.576036 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca082482-1ea0-4839-bb85-aa2b11667d91 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:42.580771 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bff8ce-327e-42cd-9bd3-804bbb7aed83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:42.581394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bff8ce-327e-42cd-9bd3-804bbb7aed83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:42.592916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bff8ce-327e-42cd-9bd3-804bbb7aed83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:42.592916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bff8ce-327e-42cd-9bd3-804bbb7aed83 admin admin] PATH_INFO: `/v3/users/7f2b8b7c42a243cfb105ef84d6bf9ed0` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:42.592916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bff8ce-327e-42cd-9bd3-804bbb7aed83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2b8b7c42a243cfb105ef84d6bf9ed0)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:42.592916 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bff8ce-327e-42cd-9bd3-804bbb7aed83 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:42.829892 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7bff8ce-327e-42cd-9bd3-804bbb7aed83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2b8b7c42a243cfb105ef84d6bf9ed0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:42.895274 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7bff8ce-327e-42cd-9bd3-804bbb7aed83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2b8b7c42a243cfb105ef84d6bf9ed0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:42.907906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7bff8ce-327e-42cd-9bd3-804bbb7aed83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2b8b7c42a243cfb105ef84d6bf9ed0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:42.908583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7bff8ce-327e-42cd-9bd3-804bbb7aed83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2b8b7c42a243cfb105ef84d6bf9ed0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:42.909261 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d7bff8ce-327e-42cd-9bd3-804bbb7aed83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2b8b7c42a243cfb105ef84d6bf9ed0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:42.921630 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1413/2780] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:42 2022] DELETE /identity/v3/users/7f2b8b7c42a243cfb105ef84d6bf9ed0 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:42.926072 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b0c19-056b-4a6d-ada9-95509724fff9 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:42.926551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b0c19-056b-4a6d-ada9-95509724fff9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:42.926651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b0c19-056b-4a6d-ada9-95509724fff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:42.946009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-ce4b0c19-056b-4a6d-ada9-95509724fff9 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:42.950868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ce4b0c19-056b-4a6d-ada9-95509724fff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:43.007400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bff8ce-327e-42cd-9bd3-804bbb7aed83 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:43.008943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d7bff8ce-327e-42cd-9bd3-804bbb7aed83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:43.013764 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1368/2781] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:43.022949 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bff8ce-327e-42cd-9bd3-804bbb7aed83 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:43.023150 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b0c19-056b-4a6d-ada9-95509724fff9 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:43.025679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ce4b0c19-056b-4a6d-ada9-95509724fff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:43.028958 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:43.029666 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:43.030039 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:43.030385 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] PATH_INFO: `/v3/projects/ed1345f3499847dbbf749a99c060271c` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:43.031599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed1345f3499847dbbf749a99c060271c)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:43.033649 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:43.090944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b0c19-056b-4a6d-ada9-95509724fff9 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:43.094637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae67b82-4909-4682-b744-c528bc02ede4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:43.095250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae67b82-4909-4682-b744-c528bc02ede4 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:43.095508 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae67b82-4909-4682-b744-c528bc02ede4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:43.095776 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae67b82-4909-4682-b744-c528bc02ede4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:43.104842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae67b82-4909-4682-b744-c528bc02ede4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:43.106805 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae67b82-4909-4682-b744-c528bc02ede4 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:43.116138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1369/2782] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:43.190426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:43.193226 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed1345f3499847dbbf749a99c060271c', 'request_id': 'req-480046ea-102e-4f3b-9df8-44db73e30fc0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.309818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed1345f3499847dbbf749a99c060271c', 'request_id': 'req-480046ea-102e-4f3b-9df8-44db73e30fc0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.311628 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed1345f3499847dbbf749a99c060271c', 'request_id': 'req-480046ea-102e-4f3b-9df8-44db73e30fc0'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.325679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] The token cache is being invalidate because project ed1345f3499847dbbf749a99c060271c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:43.326005 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.326686 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.329617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1414/2783] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:43 2022] DELETE /identity/v3/projects/ed1345f3499847dbbf749a99c060271c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:43.334143 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae67b82-4909-4682-b744-c528bc02ede4 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:43.344590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae67b82-4909-4682-b744-c528bc02ede4 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:43.417135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbfa32-54bd-4009-b60c-2f4512b6e0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:43.417842 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbfa32-54bd-4009-b60c-2f4512b6e0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:43.418123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbfa32-54bd-4009-b60c-2f4512b6e0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:43.418393 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbfa32-54bd-4009-b60c-2f4512b6e0ab admin admin] PATH_INFO: `/v3/users/3e6655f9abd74f7c8ce4e894e738b7a1` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:43.420129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbfa32-54bd-4009-b60c-2f4512b6e0ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e6655f9abd74f7c8ce4e894e738b7a1)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:43.422078 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbfa32-54bd-4009-b60c-2f4512b6e0ab admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:43.571880 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:43.571880 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:43.586311 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480046ea-102e-4f3b-9df8-44db73e30fc0 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:43.592296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:43.592910 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:43.593187 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:43.593526 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] PATH_INFO: `/v3/projects/395ace0757fa4822bdf760af3248a0b9` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:43.594792 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=395ace0757fa4822bdf760af3248a0b9)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:43.596697 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:43.629375 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:43.638579 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-65fbfa32-54bd-4009-b60c-2f4512b6e0ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e6655f9abd74f7c8ce4e894e738b7a1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.652947 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-65fbfa32-54bd-4009-b60c-2f4512b6e0ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e6655f9abd74f7c8ce4e894e738b7a1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.701643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '395ace0757fa4822bdf760af3248a0b9', 'request_id': 'req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.757929 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-65fbfa32-54bd-4009-b60c-2f4512b6e0ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e6655f9abd74f7c8ce4e894e738b7a1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.758550 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-65fbfa32-54bd-4009-b60c-2f4512b6e0ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e6655f9abd74f7c8ce4e894e738b7a1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.766444 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-65fbfa32-54bd-4009-b60c-2f4512b6e0ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e6655f9abd74f7c8ce4e894e738b7a1'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.772402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1370/2784] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:43 2022] DELETE /identity/v3/users/3e6655f9abd74f7c8ce4e894e738b7a1 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:43.803723 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '395ace0757fa4822bdf760af3248a0b9', 'request_id': 'req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.810636 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '395ace0757fa4822bdf760af3248a0b9', 'request_id': 'req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.820937 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] The token cache is being invalidate because project 395ace0757fa4822bdf760af3248a0b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:43.821166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.821826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:43.823316 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1415/2785] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:43 2022] DELETE /identity/v3/projects/395ace0757fa4822bdf760af3248a0b9 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:43.827935 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbfa32-54bd-4009-b60c-2f4512b6e0ab admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:43.829344 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-65fbfa32-54bd-4009-b60c-2f4512b6e0ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:43.894790 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbfa32-54bd-4009-b60c-2f4512b6e0ab admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:43.900271 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d31915-bc50-4e4a-93c4-bcc4f6753c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:43.900894 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d31915-bc50-4e4a-93c4-bcc4f6753c40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:43.901171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d31915-bc50-4e4a-93c4-bcc4f6753c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:43.901441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d31915-bc50-4e4a-93c4-bcc4f6753c40 admin admin] PATH_INFO: `/v3/users/5dffce967c3f43b9833effa5e7874530` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:43.902623 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d31915-bc50-4e4a-93c4-bcc4f6753c40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dffce967c3f43b9833effa5e7874530)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:43.905151 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d31915-bc50-4e4a-93c4-bcc4f6753c40 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:44.094031 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d9d31915-bc50-4e4a-93c4-bcc4f6753c40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dffce967c3f43b9833effa5e7874530'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:44.105891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d9d31915-bc50-4e4a-93c4-bcc4f6753c40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dffce967c3f43b9833effa5e7874530'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:44.191740 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d9d31915-bc50-4e4a-93c4-bcc4f6753c40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dffce967c3f43b9833effa5e7874530'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:44.192256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d9d31915-bc50-4e4a-93c4-bcc4f6753c40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dffce967c3f43b9833effa5e7874530'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:44.203732 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:44.203732 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:44.204025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d9d31915-bc50-4e4a-93c4-bcc4f6753c40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dffce967c3f43b9833effa5e7874530'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:44.210998 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1371/2786] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:43 2022] DELETE /identity/v3/users/5dffce967c3f43b9833effa5e7874530 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:44.275243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db019c5-f15b-4cb7-973a-0c2b5fab1fe9 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:44.331682 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:44.332318 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:44.332594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:44.332864 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] PATH_INFO: `/v3/projects/6b2c96cf9cba4894aff5be68c377c74c` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:44.334085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b2c96cf9cba4894aff5be68c377c74c)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:44.335963 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:44.342358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:44.467074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b2c96cf9cba4894aff5be68c377c74c', 'request_id': 'req-f2728fc4-7244-4d45-8bc7-3566e774b2a7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:44.597968 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b2c96cf9cba4894aff5be68c377c74c', 'request_id': 'req-f2728fc4-7244-4d45-8bc7-3566e774b2a7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:44.604744 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b2c96cf9cba4894aff5be68c377c74c', 'request_id': 'req-f2728fc4-7244-4d45-8bc7-3566e774b2a7'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:44.615358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] The token cache is being invalidate because project 6b2c96cf9cba4894aff5be68c377c74c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:44.615646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:44.616273 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:44.617753 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1416/2787] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:44 2022] DELETE /identity/v3/projects/6b2c96cf9cba4894aff5be68c377c74c => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:44.628019 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:44.628192 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d31915-bc50-4e4a-93c4-bcc4f6753c40 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:44.629654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:44.629786 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d9d31915-bc50-4e4a-93c4-bcc4f6753c40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:44.644327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d31915-bc50-4e4a-93c4-bcc4f6753c40 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:44.674444 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:44.702760 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2728fc4-7244-4d45-8bc7-3566e774b2a7 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:44.702982 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5a9af-ea8c-48f8-97ee-1e97904e779b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:44.703431 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5a9af-ea8c-48f8-97ee-1e97904e779b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:44.703714 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5a9af-ea8c-48f8-97ee-1e97904e779b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:44.703989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5a9af-ea8c-48f8-97ee-1e97904e779b admin admin] PATH_INFO: `/v3/users/7c5256688a7845809b227de554e09ae6` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:44.705123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5a9af-ea8c-48f8-97ee-1e97904e779b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c5256688a7845809b227de554e09ae6)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:44.707149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5a9af-ea8c-48f8-97ee-1e97904e779b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:44.707688 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d7d30-43d3-4134-9272-4f05d5e13a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:44.708314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d7d30-43d3-4134-9272-4f05d5e13a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:44.708595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d7d30-43d3-4134-9272-4f05d5e13a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:44.708866 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d7d30-43d3-4134-9272-4f05d5e13a28 admin admin] PATH_INFO: `/v3/projects/67798287cbfd4bcc951c03ab802782bc` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:44.710027 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d7d30-43d3-4134-9272-4f05d5e13a28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67798287cbfd4bcc951c03ab802782bc)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:44.711956 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52d7d30-43d3-4134-9272-4f05d5e13a28 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:44.886666 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d1b5a9af-ea8c-48f8-97ee-1e97904e779b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c5256688a7845809b227de554e09ae6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:44.886822 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b52d7d30-43d3-4134-9272-4f05d5e13a28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67798287cbfd4bcc951c03ab802782bc', 'request_id': 'req-b52d7d30-43d3-4134-9272-4f05d5e13a28'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:44.899164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d1b5a9af-ea8c-48f8-97ee-1e97904e779b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c5256688a7845809b227de554e09ae6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:45.186898 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b52d7d30-43d3-4134-9272-4f05d5e13a28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67798287cbfd4bcc951c03ab802782bc', 'request_id': 'req-b52d7d30-43d3-4134-9272-4f05d5e13a28'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:45.193954 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b52d7d30-43d3-4134-9272-4f05d5e13a28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67798287cbfd4bcc951c03ab802782bc', 'request_id': 'req-b52d7d30-43d3-4134-9272-4f05d5e13a28'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:45.204187 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b52d7d30-43d3-4134-9272-4f05d5e13a28 admin admin] The token cache is being invalidate because project 67798287cbfd4bcc951c03ab802782bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:45.204472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b52d7d30-43d3-4134-9272-4f05d5e13a28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:45.205090 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-b52d7d30-43d3-4134-9272-4f05d5e13a28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:45.206747 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1417/2788] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:44 2022] DELETE /identity/v3/projects/67798287cbfd4bcc951c03ab802782bc => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:45.211290 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291cf33-dbd2-41b7-a98e-931cf978936e None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:45.211575 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291cf33-dbd2-41b7-a98e-931cf978936e None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:45.211851 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291cf33-dbd2-41b7-a98e-931cf978936e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:45.232466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-a291cf33-dbd2-41b7-a98e-931cf978936e None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:45.233664 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-a291cf33-dbd2-41b7-a98e-931cf978936e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:45.252404 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:45.265964 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d1b5a9af-ea8c-48f8-97ee-1e97904e779b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c5256688a7845809b227de554e09ae6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:45.266569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d1b5a9af-ea8c-48f8-97ee-1e97904e779b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c5256688a7845809b227de554e09ae6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:45.274028 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d1b5a9af-ea8c-48f8-97ee-1e97904e779b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c5256688a7845809b227de554e09ae6'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:45.280151 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1372/2789] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:44 2022] DELETE /identity/v3/users/7c5256688a7845809b227de554e09ae6 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:45.292325 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1418/2790] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:45.300402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040cb12-1e8c-44f5-8cfa-4479a442d0da None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:45.300761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040cb12-1e8c-44f5-8cfa-4479a442d0da None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:45.301036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1040cb12-1e8c-44f5-8cfa-4479a442d0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:45.320876 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-1040cb12-1e8c-44f5-8cfa-4479a442d0da None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:45.322122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1040cb12-1e8c-44f5-8cfa-4479a442d0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:45.382129 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1373/2791] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:45.386521 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291cf33-dbd2-41b7-a98e-931cf978936e None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:45.396170 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291cf33-dbd2-41b7-a98e-931cf978936e None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:45.400811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a29f6-167d-4951-9293-03518e920f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:45.401403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a29f6-167d-4951-9293-03518e920f80 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:45.401693 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a29f6-167d-4951-9293-03518e920f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:45.401999 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a29f6-167d-4951-9293-03518e920f80 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:45.402626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a29f6-167d-4951-9293-03518e920f80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:04:45.402944 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a29f6-167d-4951-9293-03518e920f80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:45.404524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a29f6-167d-4951-9293-03518e920f80 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:45.415731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1419/2792] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:04:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:04:45.420565 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040cb12-1e8c-44f5-8cfa-4479a442d0da None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:45.429590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1040cb12-1e8c-44f5-8cfa-4479a442d0da None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:45.434289 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195da50-0649-4c3c-83df-092e9d67cbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:45.447830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195da50-0649-4c3c-83df-092e9d67cbc2 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:45.447830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195da50-0649-4c3c-83df-092e9d67cbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:45.447830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195da50-0649-4c3c-83df-092e9d67cbc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:45.447830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195da50-0649-4c3c-83df-092e9d67cbc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:45.447830 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195da50-0649-4c3c-83df-092e9d67cbc2 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:45.551620 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1374/2793] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:45 2022] POST /identity/v3/projects => generated 375 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:45.552761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a29f6-167d-4951-9293-03518e920f80 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:45.562675 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a29f6-167d-4951-9293-03518e920f80 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:45.619416 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97fb17-03eb-486f-8ab1-fcc62185df75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:45.620112 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97fb17-03eb-486f-8ab1-fcc62185df75 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:45.620281 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97fb17-03eb-486f-8ab1-fcc62185df75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:45.620548 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97fb17-03eb-486f-8ab1-fcc62185df75 admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:45.621161 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97fb17-03eb-486f-8ab1-fcc62185df75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:45.623140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97fb17-03eb-486f-8ab1-fcc62185df75 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:45.723650 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195da50-0649-4c3c-83df-092e9d67cbc2 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:45.725618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d195da50-0649-4c3c-83df-092e9d67cbc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:45.735482 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1420/2794] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:45 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:45.737077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97fb17-03eb-486f-8ab1-fcc62185df75 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:45.740574 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195da50-0649-4c3c-83df-092e9d67cbc2 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:45.746022 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:45.748450 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:45.748450 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:45.748450 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f admin admin] PATH_INFO: `/v3/projects/3438f0c510cf473a93c2bd0e95f669d6` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:45.748770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97fb17-03eb-486f-8ab1-fcc62185df75 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:45.748887 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3438f0c510cf473a93c2bd0e95f669d6)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:45.750357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:45.752732 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a7de4-e983-40a9-bc74-2589c7616eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:45.753322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a7de4-e983-40a9-bc74-2589c7616eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:45.753595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a7de4-e983-40a9-bc74-2589c7616eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:45.753899 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a7de4-e983-40a9-bc74-2589c7616eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:45.754609 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a7de4-e983-40a9-bc74-2589c7616eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:45.756148 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a7de4-e983-40a9-bc74-2589c7616eb8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:45.767623 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1421/2795] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:45.771339 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a7de4-e983-40a9-bc74-2589c7616eb8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:45.776631 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:45.781409 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a7de4-e983-40a9-bc74-2589c7616eb8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:45.785994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbfe43-0fef-433b-ab8a-fd1c7cb6626a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:45.786565 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbfe43-0fef-433b-ab8a-fd1c7cb6626a admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:45.786838 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbfe43-0fef-433b-ab8a-fd1c7cb6626a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:45.787103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbfe43-0fef-433b-ab8a-fd1c7cb6626a admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:45.787738 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbfe43-0fef-433b-ab8a-fd1c7cb6626a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:45.789293 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbfe43-0fef-433b-ab8a-fd1c7cb6626a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:45.800595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1422/2796] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:45.804520 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbfe43-0fef-433b-ab8a-fd1c7cb6626a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:45.813782 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbfe43-0fef-433b-ab8a-fd1c7cb6626a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:45.818403 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab56765a-af63-41f9-bbcc-4349febc0515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:45.821883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56765a-af63-41f9-bbcc-4349febc0515 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:45.821883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56765a-af63-41f9-bbcc-4349febc0515 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:45.821883 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab56765a-af63-41f9-bbcc-4349febc0515 admin admin] PATH_INFO: `/v3/projects/9eb151cd9f624b72a34d419f94be1d17/users/19b6fc0115c747a19f5aa2603d926632/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:45.833374 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab56765a-af63-41f9-bbcc-4349febc0515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb151cd9f624b72a34d419f94be1d17, user_id=19b6fc0115c747a19f5aa2603d926632, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:45.836051 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab56765a-af63-41f9-bbcc-4349febc0515 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:45.853887 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3438f0c510cf473a93c2bd0e95f669d6', 'request_id': 'req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:45.860384 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3438f0c510cf473a93c2bd0e95f669d6', 'request_id': 'req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:45.866850 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3438f0c510cf473a93c2bd0e95f669d6', 'request_id': 'req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:45.900900 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1423/2797] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:45 2022] PUT /identity/v3/projects/9eb151cd9f624b72a34d419f94be1d17/users/19b6fc0115c747a19f5aa2603d926632/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:45.911642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9d6ce-510c-4643-95a7-6a673772b4cf None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:45.911927 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9d6ce-510c-4643-95a7-6a673772b4cf None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:45.912203 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f9d6ce-510c-4643-95a7-6a673772b4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:45.940248 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-d2f9d6ce-510c-4643-95a7-6a673772b4cf None None] MFA Rules not processed for user `19b6fc0115c747a19f5aa2603d926632`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:45.941558 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d2f9d6ce-510c-4643-95a7-6a673772b4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:45.969185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f admin admin] The token cache is being invalidate because project 3438f0c510cf473a93c2bd0e95f669d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:45.969460 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:45.970128 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d2cc6f89-cfee-474f-aec6-2a4ab3e5d99f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:45.971945 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1375/2798] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:45 2022] DELETE /identity/v3/projects/3438f0c510cf473a93c2bd0e95f669d6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:45.976532 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cd04e-b4d7-4a0b-ac8d-bab59b0a2706 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:45.976818 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cd04e-b4d7-4a0b-ac8d-bab59b0a2706 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:45.977095 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7cd04e-b4d7-4a0b-ac8d-bab59b0a2706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:45.989152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:45.997323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-3c7cd04e-b4d7-4a0b-ac8d-bab59b0a2706 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:45.998564 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-3c7cd04e-b4d7-4a0b-ac8d-bab59b0a2706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:46.009462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1424/2799] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:46.014410 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9d6ce-510c-4643-95a7-6a673772b4cf None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:46.015789 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d2f9d6ce-510c-4643-95a7-6a673772b4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:46.057726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1376/2800] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:46.065654 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a56c0-8571-424d-939a-7b16c35ad4aa None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.065976 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a56c0-8571-424d-939a-7b16c35ad4aa None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.066255 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a56c0-8571-424d-939a-7b16c35ad4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.079765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f9d6ce-510c-4643-95a7-6a673772b4cf None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:46.084462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b4ae-6637-492d-9bb5-3d1b122a0501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:46.085058 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b4ae-6637-492d-9bb5-3d1b122a0501 admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.085331 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b4ae-6637-492d-9bb5-3d1b122a0501 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.085599 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa59b4ae-6637-492d-9bb5-3d1b122a0501 admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.085712 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-285a56c0-8571-424d-939a-7b16c35ad4aa None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:46.086241 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59b4ae-6637-492d-9bb5-3d1b122a0501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:46.086868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-285a56c0-8571-424d-939a-7b16c35ad4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:46.088461 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa59b4ae-6637-492d-9bb5-3d1b122a0501 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:46.096368 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1377/2801] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:46.099712 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a56c0-8571-424d-939a-7b16c35ad4aa None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:46.109376 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a56c0-8571-424d-939a-7b16c35ad4aa None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:46.114244 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904d6ed-3a0f-4310-b826-3bc0babb0b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:46.114827 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904d6ed-3a0f-4310-b826-3bc0babb0b08 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.115104 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904d6ed-3a0f-4310-b826-3bc0babb0b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.115378 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904d6ed-3a0f-4310-b826-3bc0babb0b08 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.116008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c904d6ed-3a0f-4310-b826-3bc0babb0b08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:04:46.116308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c904d6ed-3a0f-4310-b826-3bc0babb0b08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:46.117951 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c904d6ed-3a0f-4310-b826-3bc0babb0b08 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:46.129968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1378/2802] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 20:04:46.134200 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904d6ed-3a0f-4310-b826-3bc0babb0b08 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:46.143495 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904d6ed-3a0f-4310-b826-3bc0babb0b08 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:46.148181 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e1c83-639e-40eb-8854-e6a61c119413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:46.148727 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e1c83-639e-40eb-8854-e6a61c119413 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.149009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e1c83-639e-40eb-8854-e6a61c119413 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.149277 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002e1c83-639e-40eb-8854-e6a61c119413 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.149903 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e1c83-639e-40eb-8854-e6a61c119413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:46.151735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002e1c83-639e-40eb-8854-e6a61c119413 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:46.199356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1425/2803] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:46 2022] POST /identity/v3/projects => generated 375 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:46.203194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b4ae-6637-492d-9bb5-3d1b122a0501 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:46.264714 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa59b4ae-6637-492d-9bb5-3d1b122a0501 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:46.269366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd260fa-9768-40ae-be92-a2970817c7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:46.269994 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd260fa-9768-40ae-be92-a2970817c7af admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.270271 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd260fa-9768-40ae-be92-a2970817c7af admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.270541 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd260fa-9768-40ae-be92-a2970817c7af admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.271155 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd260fa-9768-40ae-be92-a2970817c7af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:46.273049 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd260fa-9768-40ae-be92-a2970817c7af admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:46.285587 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1379/2804] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:46 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:46.289780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e1c83-639e-40eb-8854-e6a61c119413 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:46.350906 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002e1c83-639e-40eb-8854-e6a61c119413 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:46.355700 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09f969-c2c5-4b05-ac5f-d952c187175f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:46.356360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09f969-c2c5-4b05-ac5f-d952c187175f admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.357149 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09f969-c2c5-4b05-ac5f-d952c187175f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.357422 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb09f969-c2c5-4b05-ac5f-d952c187175f admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.358069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09f969-c2c5-4b05-ac5f-d952c187175f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:46.360008 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb09f969-c2c5-4b05-ac5f-d952c187175f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:46.367550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1426/2805] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:46 2022] POST /identity/v3/users => generated 294 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:46.371445 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd260fa-9768-40ae-be92-a2970817c7af admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:46.380871 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd260fa-9768-40ae-be92-a2970817c7af admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:46.385442 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb08f2a-23e8-4f27-b9e4-018083db6e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:46.386037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb08f2a-23e8-4f27-b9e4-018083db6e3d admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.386314 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb08f2a-23e8-4f27-b9e4-018083db6e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.386579 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb08f2a-23e8-4f27-b9e4-018083db6e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.387205 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb08f2a-23e8-4f27-b9e4-018083db6e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:46.388799 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb08f2a-23e8-4f27-b9e4-018083db6e3d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:46.399260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1427/2806] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:46.403759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb08f2a-23e8-4f27-b9e4-018083db6e3d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:46.413145 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb08f2a-23e8-4f27-b9e4-018083db6e3d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:46.417882 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e698f36-5b44-45e8-ba07-0a9b584281e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:46.418477 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e698f36-5b44-45e8-ba07-0a9b584281e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.419444 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e698f36-5b44-45e8-ba07-0a9b584281e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.419715 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e698f36-5b44-45e8-ba07-0a9b584281e8 admin admin] PATH_INFO: `/v3/projects/1dd97fe017264e0da75bff46c6adb338/users/15ddf148d2b54f19b1b8867add56cc11/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.433829 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e698f36-5b44-45e8-ba07-0a9b584281e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd97fe017264e0da75bff46c6adb338, user_id=15ddf148d2b54f19b1b8867add56cc11, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:46.436429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e698f36-5b44-45e8-ba07-0a9b584281e8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:46.454425 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:46.459692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1380/2807] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:46 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:46.464036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09f969-c2c5-4b05-ac5f-d952c187175f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:46.474005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb09f969-c2c5-4b05-ac5f-d952c187175f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:46.478606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8fae3-10ae-4034-a0dc-fa08b46b7aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:46.479179 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8fae3-10ae-4034-a0dc-fa08b46b7aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.479455 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8fae3-10ae-4034-a0dc-fa08b46b7aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.479722 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8fae3-10ae-4034-a0dc-fa08b46b7aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.480336 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8fae3-10ae-4034-a0dc-fa08b46b7aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:46.482622 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8fae3-10ae-4034-a0dc-fa08b46b7aa8 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:46.492285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1381/2808] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:46.503127 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8fae3-10ae-4034-a0dc-fa08b46b7aa8 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:46.505532 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8fae3-10ae-4034-a0dc-fa08b46b7aa8 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:46.510208 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4364f-938b-4425-b126-049fcf80d1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:46.510663 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4364f-938b-4425-b126-049fcf80d1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.510942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4364f-938b-4425-b126-049fcf80d1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.511213 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4364f-938b-4425-b126-049fcf80d1e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.511821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4364f-938b-4425-b126-049fcf80d1e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:46.513346 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4364f-938b-4425-b126-049fcf80d1e9 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:46.532596 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1382/2809] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:46.532786 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1428/2810] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:46 2022] PUT /identity/v3/projects/1dd97fe017264e0da75bff46c6adb338/users/15ddf148d2b54f19b1b8867add56cc11/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:46.534249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4364f-938b-4425-b126-049fcf80d1e9 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:46.534529 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e698f36-5b44-45e8-ba07-0a9b584281e8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:46.544440 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4364f-938b-4425-b126-049fcf80d1e9 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:46.561498 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:46.595831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e698f36-5b44-45e8-ba07-0a9b584281e8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:46.600510 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ec143-c7ff-4c03-a44f-8699f39da7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:46.623389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26feff85-7eea-44f5-ab4e-df4fa889320f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:46.623389 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26feff85-7eea-44f5-ab4e-df4fa889320f admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.623675 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ec143-c7ff-4c03-a44f-8699f39da7e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.624169 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26feff85-7eea-44f5-ab4e-df4fa889320f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.624278 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ec143-c7ff-4c03-a44f-8699f39da7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.624386 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26feff85-7eea-44f5-ab4e-df4fa889320f admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.624496 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970ec143-c7ff-4c03-a44f-8699f39da7e8 admin admin] PATH_INFO: `/v3/projects/e8a911fb157d42ae9c4adb6442bc982e/users/edc59f2f06574efca3a2114ef57b112c/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.624610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26feff85-7eea-44f5-ab4e-df4fa889320f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:46.624610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26feff85-7eea-44f5-ab4e-df4fa889320f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:46.624610 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1383/2811] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:46.624894 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ec143-c7ff-4c03-a44f-8699f39da7e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a911fb157d42ae9c4adb6442bc982e, user_id=edc59f2f06574efca3a2114ef57b112c, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:46.624894 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970ec143-c7ff-4c03-a44f-8699f39da7e8 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:46.626522 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26feff85-7eea-44f5-ab4e-df4fa889320f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:46.636335 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26feff85-7eea-44f5-ab4e-df4fa889320f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:46.641099 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34ac06-f964-49bf-a871-998aaef6c695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:46.641771 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34ac06-f964-49bf-a871-998aaef6c695 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.642080 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34ac06-f964-49bf-a871-998aaef6c695 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.642361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34ac06-f964-49bf-a871-998aaef6c695 admin admin] PATH_INFO: `/v3/projects/1dd97fe017264e0da75bff46c6adb338/users/15ddf148d2b54f19b1b8867add56cc11/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.644321 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34ac06-f964-49bf-a871-998aaef6c695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dd97fe017264e0da75bff46c6adb338, user_id=15ddf148d2b54f19b1b8867add56cc11, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:46.647037 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34ac06-f964-49bf-a871-998aaef6c695 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:46.657534 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1384/2812] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:46 2022] PUT /identity/v3/projects/1dd97fe017264e0da75bff46c6adb338/users/15ddf148d2b54f19b1b8867add56cc11/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:46.668627 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de6def-bbb3-43cd-aed5-a172cc926018 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.669458 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de6def-bbb3-43cd-aed5-a172cc926018 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.669731 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de6def-bbb3-43cd-aed5-a172cc926018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.697049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-42de6def-bbb3-43cd-aed5-a172cc926018 None None] MFA Rules not processed for user `15ddf148d2b54f19b1b8867add56cc11`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:46.698329 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-42de6def-bbb3-43cd-aed5-a172cc926018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:46.717014 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1429/2813] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:46 2022] PUT /identity/v3/projects/e8a911fb157d42ae9c4adb6442bc982e/users/edc59f2f06574efca3a2114ef57b112c/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:46.772086 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1385/2814] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:46.784917 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ec143-c7ff-4c03-a44f-8699f39da7e8 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:46.786133 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92cdc7-bc04-419f-9169-9a2c87be3c07 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.786419 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92cdc7-bc04-419f-9169-9a2c87be3c07 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.786629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-970ec143-c7ff-4c03-a44f-8699f39da7e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:46.786755 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92cdc7-bc04-419f-9169-9a2c87be3c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.815693 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-6d92cdc7-bc04-419f-9169-9a2c87be3c07 None None] MFA Rules not processed for user `edc59f2f06574efca3a2114ef57b112c`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:46.816952 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6d92cdc7-bc04-419f-9169-9a2c87be3c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:46.846821 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970ec143-c7ff-4c03-a44f-8699f39da7e8 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:46.850589 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b13e34-1ebc-408f-b3d4-a75ca9f19e8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:46.851194 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b13e34-1ebc-408f-b3d4-a75ca9f19e8f service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.851468 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b13e34-1ebc-408f-b3d4-a75ca9f19e8f service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.852318 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b13e34-1ebc-408f-b3d4-a75ca9f19e8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.852447 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:46.884293 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1386/2815] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:46.896243 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92cdc7-bc04-419f-9169-9a2c87be3c07 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:46.897645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-6d92cdc7-bc04-419f-9169-9a2c87be3c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:46.910052 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b13e34-1ebc-408f-b3d4-a75ca9f19e8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:46.910527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92cdc7-bc04-419f-9169-9a2c87be3c07 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:46.911561 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b13e34-1ebc-408f-b3d4-a75ca9f19e8f service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:46.914761 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2898b00-58aa-4bef-9c8b-d0b190897775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:46.915363 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2898b00-58aa-4bef-9c8b-d0b190897775 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:46.915640 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2898b00-58aa-4bef-9c8b-d0b190897775 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:46.915911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2898b00-58aa-4bef-9c8b-d0b190897775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:46.920745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1430/2816] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:46.924821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2898b00-58aa-4bef-9c8b-d0b190897775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:46.926359 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2898b00-58aa-4bef-9c8b-d0b190897775 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:46.935370 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1387/2817] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:50.052660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b13e34-1ebc-408f-b3d4-a75ca9f19e8f service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:50.113058 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b13e34-1ebc-408f-b3d4-a75ca9f19e8f service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:50.117598 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50e8574-7717-4fba-b656-3d206d7a5142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:50.118242 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e8574-7717-4fba-b656-3d206d7a5142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:50.118518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e8574-7717-4fba-b656-3d206d7a5142 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:50.118786 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50e8574-7717-4fba-b656-3d206d7a5142 admin admin] PATH_INFO: `/v3/users/19b6fc0115c747a19f5aa2603d926632` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:50.119872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e8574-7717-4fba-b656-3d206d7a5142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19b6fc0115c747a19f5aa2603d926632)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:50.121764 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50e8574-7717-4fba-b656-3d206d7a5142 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:50.123976 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:50.235946 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2898b00-58aa-4bef-9c8b-d0b190897775 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:50.246655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2898b00-58aa-4bef-9c8b-d0b190897775 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:50.251141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca942a83-3de2-491e-ba35-b5235989cbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:50.251754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca942a83-3de2-491e-ba35-b5235989cbdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:50.252033 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca942a83-3de2-491e-ba35-b5235989cbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:50.252306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca942a83-3de2-491e-ba35-b5235989cbdf admin admin] PATH_INFO: `/v3/users/edc59f2f06574efca3a2114ef57b112c` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:50.253468 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca942a83-3de2-491e-ba35-b5235989cbdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=edc59f2f06574efca3a2114ef57b112c)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:50.255414 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca942a83-3de2-491e-ba35-b5235989cbdf admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:50.342711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d50e8574-7717-4fba-b656-3d206d7a5142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19b6fc0115c747a19f5aa2603d926632'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:50.443384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d50e8574-7717-4fba-b656-3d206d7a5142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19b6fc0115c747a19f5aa2603d926632'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:50.455918 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d50e8574-7717-4fba-b656-3d206d7a5142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19b6fc0115c747a19f5aa2603d926632'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:50.456573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d50e8574-7717-4fba-b656-3d206d7a5142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19b6fc0115c747a19f5aa2603d926632'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:50.457172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d50e8574-7717-4fba-b656-3d206d7a5142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19b6fc0115c747a19f5aa2603d926632'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:50.470123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1431/2818] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:50 2022] DELETE /identity/v3/users/19b6fc0115c747a19f5aa2603d926632 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:50.474357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4d465-71d9-4436-9594-a02a82f582d4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:50.474656 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4d465-71d9-4436-9594-a02a82f582d4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:50.474915 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4d465-71d9-4436-9594-a02a82f582d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:50.486708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ca942a83-3de2-491e-ba35-b5235989cbdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edc59f2f06574efca3a2114ef57b112c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:50.494139 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7ba4d465-71d9-4436-9594-a02a82f582d4 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:50.495356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7ba4d465-71d9-4436-9594-a02a82f582d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:50.498943 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ca942a83-3de2-491e-ba35-b5235989cbdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edc59f2f06574efca3a2114ef57b112c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:50.553366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1432/2819] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:50.563142 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4d465-71d9-4436-9594-a02a82f582d4 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:50.564415 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7ba4d465-71d9-4436-9594-a02a82f582d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:50.601395 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ca942a83-3de2-491e-ba35-b5235989cbdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edc59f2f06574efca3a2114ef57b112c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:50.602063 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ca942a83-3de2-491e-ba35-b5235989cbdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edc59f2f06574efca3a2114ef57b112c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:50.609694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-ca942a83-3de2-491e-ba35-b5235989cbdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edc59f2f06574efca3a2114ef57b112c'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:50.615873 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1388/2820] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:50 2022] DELETE /identity/v3/users/edc59f2f06574efca3a2114ef57b112c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:50.620569 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a31c2f-c50e-49a1-b569-15d8f922ebe4 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:50.620700 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a31c2f-c50e-49a1-b569-15d8f922ebe4 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:50.620887 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a31c2f-c50e-49a1-b569-15d8f922ebe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:50.621072 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4d465-71d9-4436-9594-a02a82f582d4 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:50.640109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-07a31c2f-c50e-49a1-b569-15d8f922ebe4 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:50.641460 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-07a31c2f-c50e-49a1-b569-15d8f922ebe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:50.656166 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:50.671524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980912c5-edc7-4614-bb26-84542a45c051 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:50.672151 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980912c5-edc7-4614-bb26-84542a45c051 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:50.672423 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980912c5-edc7-4614-bb26-84542a45c051 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:50.672687 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980912c5-edc7-4614-bb26-84542a45c051 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:50.674451 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-980912c5-edc7-4614-bb26-84542a45c051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:50.700411 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1389/2821] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:50.710220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a31c2f-c50e-49a1-b569-15d8f922ebe4 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:50.711588 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-07a31c2f-c50e-49a1-b569-15d8f922ebe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:50.725248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a31c2f-c50e-49a1-b569-15d8f922ebe4 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:50.728219 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1299fd0-3d3b-4a43-ab38-b4ff44cfc389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:50.728813 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1299fd0-3d3b-4a43-ab38-b4ff44cfc389 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:50.729090 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1299fd0-3d3b-4a43-ab38-b4ff44cfc389 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:50.729357 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1299fd0-3d3b-4a43-ab38-b4ff44cfc389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:50.738417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1299fd0-3d3b-4a43-ab38-b4ff44cfc389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:50.738585 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980912c5-edc7-4614-bb26-84542a45c051 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:50.740486 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1299fd0-3d3b-4a43-ab38-b4ff44cfc389 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:50.740681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980912c5-edc7-4614-bb26-84542a45c051 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:50.749741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1390/2822] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:04:50.750574 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1433/2823] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:04:51.314339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1299fd0-3d3b-4a43-ab38-b4ff44cfc389 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:51.315856 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a1299fd0-3d3b-4a43-ab38-b4ff44cfc389 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:51.329198 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1299fd0-3d3b-4a43-ab38-b4ff44cfc389 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:51.333896 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294ac80-951d-40c1-a220-5fcf6d397db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:51.334524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294ac80-951d-40c1-a220-5fcf6d397db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:51.334796 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294ac80-951d-40c1-a220-5fcf6d397db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:51.335091 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294ac80-951d-40c1-a220-5fcf6d397db0 admin admin] PATH_INFO: `/v3/projects/e8a911fb157d42ae9c4adb6442bc982e` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:51.336173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294ac80-951d-40c1-a220-5fcf6d397db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8a911fb157d42ae9c4adb6442bc982e)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:51.338416 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294ac80-951d-40c1-a220-5fcf6d397db0 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:51.349511 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980912c5-edc7-4614-bb26-84542a45c051 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:51.351411 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-980912c5-edc7-4614-bb26-84542a45c051 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:51.367993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980912c5-edc7-4614-bb26-84542a45c051 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:51.370432 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:51.378081 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:51.378081 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:51.378081 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:51.378390 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] PATH_INFO: `/v3/projects/9eb151cd9f624b72a34d419f94be1d17` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:51.379430 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb151cd9f624b72a34d419f94be1d17)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:51.381391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:51.472190 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d294ac80-951d-40c1-a220-5fcf6d397db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8a911fb157d42ae9c4adb6442bc982e', 'request_id': 'req-d294ac80-951d-40c1-a220-5fcf6d397db0'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:51.477919 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d294ac80-951d-40c1-a220-5fcf6d397db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8a911fb157d42ae9c4adb6442bc982e', 'request_id': 'req-d294ac80-951d-40c1-a220-5fcf6d397db0'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:51.485203 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d294ac80-951d-40c1-a220-5fcf6d397db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8a911fb157d42ae9c4adb6442bc982e', 'request_id': 'req-d294ac80-951d-40c1-a220-5fcf6d397db0'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:51.496357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb151cd9f624b72a34d419f94be1d17', 'request_id': 'req-f5cffa65-6312-4a9e-923e-e8d00119f327'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:51.597645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d294ac80-951d-40c1-a220-5fcf6d397db0 admin admin] The token cache is being invalidate because project e8a911fb157d42ae9c4adb6442bc982e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:51.597893 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d294ac80-951d-40c1-a220-5fcf6d397db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:51.598530 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d294ac80-951d-40c1-a220-5fcf6d397db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:51.600021 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1391/2824] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:51 2022] DELETE /identity/v3/projects/e8a911fb157d42ae9c4adb6442bc982e => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:51.604606 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272517-8e00-417f-bb6f-4ef829c33a04 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:51.604886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272517-8e00-417f-bb6f-4ef829c33a04 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:51.605159 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5272517-8e00-417f-bb6f-4ef829c33a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:51.626019 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-a5272517-8e00-417f-bb6f-4ef829c33a04 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:51.632323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-a5272517-8e00-417f-bb6f-4ef829c33a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:51.643576 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb151cd9f624b72a34d419f94be1d17', 'request_id': 'req-f5cffa65-6312-4a9e-923e-e8d00119f327'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:51.650485 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb151cd9f624b72a34d419f94be1d17', 'request_id': 'req-f5cffa65-6312-4a9e-923e-e8d00119f327'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:51.662152 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] The token cache is being invalidate because project 9eb151cd9f624b72a34d419f94be1d17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:51.662446 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:51.663092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:51.664581 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1434/2825] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:51 2022] DELETE /identity/v3/projects/9eb151cd9f624b72a34d419f94be1d17 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:51.668511 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:51.669854 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:51.688548 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1392/2826] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:51.696270 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86178c1-0223-4a3b-b2e1-e47f30b2fd3a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:51.696550 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86178c1-0223-4a3b-b2e1-e47f30b2fd3a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:51.696819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86178c1-0223-4a3b-b2e1-e47f30b2fd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:51.716163 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-d86178c1-0223-4a3b-b2e1-e47f30b2fd3a None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:51.718131 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d86178c1-0223-4a3b-b2e1-e47f30b2fd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:51.726984 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1393/2827] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:51.731025 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86178c1-0223-4a3b-b2e1-e47f30b2fd3a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:51.738750 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cffa65-6312-4a9e-923e-e8d00119f327 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:51.740315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86178c1-0223-4a3b-b2e1-e47f30b2fd3a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:51.743549 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e62d12-0a0f-4448-ba51-e0797c47d70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:51.744156 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e62d12-0a0f-4448-ba51-e0797c47d70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:51.744433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e62d12-0a0f-4448-ba51-e0797c47d70d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:51.744702 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e62d12-0a0f-4448-ba51-e0797c47d70d admin admin] PATH_INFO: `/v3/users/15ddf148d2b54f19b1b8867add56cc11` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:51.744911 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b8686-8574-473e-97ba-f8c5f5d76c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:51.745510 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8686-8574-473e-97ba-f8c5f5d76c55 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:51.745783 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8686-8574-473e-97ba-f8c5f5d76c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:51.745929 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e62d12-0a0f-4448-ba51-e0797c47d70d admin admin] RBAC: Authorizing `identity:delete_user(user_id=15ddf148d2b54f19b1b8867add56cc11)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:51.746082 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5b8686-8574-473e-97ba-f8c5f5d76c55 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:51.746704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b8686-8574-473e-97ba-f8c5f5d76c55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:04:51.746998 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b8686-8574-473e-97ba-f8c5f5d76c55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:51.747774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e62d12-0a0f-4448-ba51-e0797c47d70d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:51.748580 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5b8686-8574-473e-97ba-f8c5f5d76c55 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:51.760233 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1394/2828] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 20:04:51.761550 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:51.764417 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b8686-8574-473e-97ba-f8c5f5d76c55 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:51.773968 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5b8686-8574-473e-97ba-f8c5f5d76c55 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:51.778567 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dc0c8-d78c-402c-8365-e7edbec41901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:51.779138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc0c8-d78c-402c-8365-e7edbec41901 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:51.779932 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc0c8-d78c-402c-8365-e7edbec41901 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:51.780203 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95dc0c8-d78c-402c-8365-e7edbec41901 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:51.780809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dc0c8-d78c-402c-8365-e7edbec41901 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:51.782698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95dc0c8-d78c-402c-8365-e7edbec41901 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:51.919229 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1395/2829] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:51 2022] POST /identity/v3/projects => generated 367 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:51.923331 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dc0c8-d78c-402c-8365-e7edbec41901 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:51.963624 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-83e62d12-0a0f-4448-ba51-e0797c47d70d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15ddf148d2b54f19b1b8867add56cc11'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:51.986215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95dc0c8-d78c-402c-8365-e7edbec41901 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:51.990858 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a55a50-03db-48d4-8922-48efa818b654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:51.991456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a55a50-03db-48d4-8922-48efa818b654 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:51.991736 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a55a50-03db-48d4-8922-48efa818b654 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:51.992005 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a55a50-03db-48d4-8922-48efa818b654 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:51.992637 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a55a50-03db-48d4-8922-48efa818b654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:51.994581 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a55a50-03db-48d4-8922-48efa818b654 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:52.004353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:52.075523 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-83e62d12-0a0f-4448-ba51-e0797c47d70d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15ddf148d2b54f19b1b8867add56cc11'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:52.089412 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-83e62d12-0a0f-4448-ba51-e0797c47d70d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15ddf148d2b54f19b1b8867add56cc11'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:52.090096 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-83e62d12-0a0f-4448-ba51-e0797c47d70d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15ddf148d2b54f19b1b8867add56cc11'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:52.090706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-83e62d12-0a0f-4448-ba51-e0797c47d70d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15ddf148d2b54f19b1b8867add56cc11'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:52.103285 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1435/2830] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:04:51 2022] DELETE /identity/v3/users/15ddf148d2b54f19b1b8867add56cc11 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:04:52.123902 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1396/2831] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:51 2022] POST /identity/v3/users => generated 284 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:52.128140 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e62d12-0a0f-4448-ba51-e0797c47d70d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:52.129471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-83e62d12-0a0f-4448-ba51-e0797c47d70d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:52.196307 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e62d12-0a0f-4448-ba51-e0797c47d70d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:52.201195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5678a-35d5-4bf3-afe1-0a9534761436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:52.202371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5678a-35d5-4bf3-afe1-0a9534761436 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:52.202648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5678a-35d5-4bf3-afe1-0a9534761436 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:52.202920 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5678a-35d5-4bf3-afe1-0a9534761436 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:52.203555 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5678a-35d5-4bf3-afe1-0a9534761436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:52.205133 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5678a-35d5-4bf3-afe1-0a9534761436 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:52.215391 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1436/2832] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:52.220122 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a55a50-03db-48d4-8922-48efa818b654 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:52.230009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a55a50-03db-48d4-8922-48efa818b654 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:52.234587 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da3787b-01c4-4176-91f6-e9c0c78cf18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:52.235172 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3787b-01c4-4176-91f6-e9c0c78cf18d admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:52.235447 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3787b-01c4-4176-91f6-e9c0c78cf18d admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:52.235717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da3787b-01c4-4176-91f6-e9c0c78cf18d admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:52.236342 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3787b-01c4-4176-91f6-e9c0c78cf18d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:52.237950 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da3787b-01c4-4176-91f6-e9c0c78cf18d admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:52.247502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1397/2833] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:52.252184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5678a-35d5-4bf3-afe1-0a9534761436 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:52.262160 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5678a-35d5-4bf3-afe1-0a9534761436 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:52.267304 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04ca66-798f-4bb3-81e9-1bccc86d7aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:52.267933 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04ca66-798f-4bb3-81e9-1bccc86d7aad admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:52.268206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04ca66-798f-4bb3-81e9-1bccc86d7aad admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:52.268474 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04ca66-798f-4bb3-81e9-1bccc86d7aad admin admin] PATH_INFO: `/v3/projects/0f467720ddbc4fb08524573719b2e7ad/users/48dfaf9f066045c2b80f4f13a0582f6e/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:52.283171 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04ca66-798f-4bb3-81e9-1bccc86d7aad admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f467720ddbc4fb08524573719b2e7ad, user_id=48dfaf9f066045c2b80f4f13a0582f6e, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:52.285924 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04ca66-798f-4bb3-81e9-1bccc86d7aad admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:52.387296 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1437/2834] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:52 2022] PUT /identity/v3/projects/0f467720ddbc4fb08524573719b2e7ad/users/48dfaf9f066045c2b80f4f13a0582f6e/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:52.398323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63253aa-0edb-49ab-9414-fae0b172873a None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:52.403624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63253aa-0edb-49ab-9414-fae0b172873a None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:52.403624 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63253aa-0edb-49ab-9414-fae0b172873a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:52.427482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-c63253aa-0edb-49ab-9414-fae0b172873a None None] MFA Rules not processed for user `48dfaf9f066045c2b80f4f13a0582f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:52.428756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c63253aa-0edb-49ab-9414-fae0b172873a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:52.495310 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1398/2835] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:52.507874 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04ca66-798f-4bb3-81e9-1bccc86d7aad admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:52.509629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8b04ca66-798f-4bb3-81e9-1bccc86d7aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:52.557874 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63253aa-0edb-49ab-9414-fae0b172873a None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:52.561775 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c63253aa-0edb-49ab-9414-fae0b172873a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:52.571328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04ca66-798f-4bb3-81e9-1bccc86d7aad admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:52.575873 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905536c6-681e-4c03-9e47-193c66108311 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e021a00524341359bc4b5a800a9bbf6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:52.577077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905536c6-681e-4c03-9e47-193c66108311 service glance] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:52.577351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905536c6-681e-4c03-9e47-193c66108311 service glance] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:52.577613 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905536c6-681e-4c03-9e47-193c66108311 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:52.588318 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905536c6-681e-4c03-9e47-193c66108311 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:52.596552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905536c6-681e-4c03-9e47-193c66108311 service glance] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:52.601082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1438/2836] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:52.632361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63253aa-0edb-49ab-9414-fae0b172873a None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:52.638186 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa7ccd-dd49-4023-b82c-21ce19fa6608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:52.638779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa7ccd-dd49-4023-b82c-21ce19fa6608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:52.639758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa7ccd-dd49-4023-b82c-21ce19fa6608 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:52.640051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa7ccd-dd49-4023-b82c-21ce19fa6608 admin admin] PATH_INFO: `/v3/projects/1dd97fe017264e0da75bff46c6adb338` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:52.641209 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa7ccd-dd49-4023-b82c-21ce19fa6608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dd97fe017264e0da75bff46c6adb338)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:52.643316 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa7ccd-dd49-4023-b82c-21ce19fa6608 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:52.751170 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-23aa7ccd-dd49-4023-b82c-21ce19fa6608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dd97fe017264e0da75bff46c6adb338', 'request_id': 'req-23aa7ccd-dd49-4023-b82c-21ce19fa6608'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:52.757006 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-23aa7ccd-dd49-4023-b82c-21ce19fa6608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dd97fe017264e0da75bff46c6adb338', 'request_id': 'req-23aa7ccd-dd49-4023-b82c-21ce19fa6608'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:52.763773 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-23aa7ccd-dd49-4023-b82c-21ce19fa6608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dd97fe017264e0da75bff46c6adb338', 'request_id': 'req-23aa7ccd-dd49-4023-b82c-21ce19fa6608'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:52.840818 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-23aa7ccd-dd49-4023-b82c-21ce19fa6608 admin admin] The token cache is being invalidate because project 1dd97fe017264e0da75bff46c6adb338 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:04:52.841110 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-23aa7ccd-dd49-4023-b82c-21ce19fa6608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:52.841740 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-23aa7ccd-dd49-4023-b82c-21ce19fa6608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:04:52.843462 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1399/2837] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:04:52 2022] DELETE /identity/v3/projects/1dd97fe017264e0da75bff46c6adb338 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:52.848252 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d803a-8163-4498-9dad-d5768ba627a3 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:52.848543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d803a-8163-4498-9dad-d5768ba627a3 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:52.852569 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d803a-8163-4498-9dad-d5768ba627a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:52.859380 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:52.871748 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-fb7d803a-8163-4498-9dad-d5768ba627a3 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:52.872954 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fb7d803a-8163-4498-9dad-d5768ba627a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:52.931795 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1439/2838] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:52.940138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f6b2e-85ee-4804-b731-2aa6d513ea73 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:52.940423 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f6b2e-85ee-4804-b731-2aa6d513ea73 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:52.940700 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f6b2e-85ee-4804-b731-2aa6d513ea73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:52.948319 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:52.960920 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-f05f6b2e-85ee-4804-b731-2aa6d513ea73 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:52.962248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-f05f6b2e-85ee-4804-b731-2aa6d513ea73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:52.970836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1400/2839] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:52.975267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d803a-8163-4498-9dad-d5768ba627a3 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:52.985466 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d803a-8163-4498-9dad-d5768ba627a3 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:52.990665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0684df-da69-4cc8-907d-3a821069fe06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:52.996346 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0684df-da69-4cc8-907d-3a821069fe06 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:52.996346 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0684df-da69-4cc8-907d-3a821069fe06 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:52.996346 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0684df-da69-4cc8-907d-3a821069fe06 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:52.996346 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0684df-da69-4cc8-907d-3a821069fe06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:04:52.996346 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0684df-da69-4cc8-907d-3a821069fe06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:52.996346 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0684df-da69-4cc8-907d-3a821069fe06 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:53.006373 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1440/2840] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 20:04:53.010709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f6b2e-85ee-4804-b731-2aa6d513ea73 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:53.019694 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f6b2e-85ee-4804-b731-2aa6d513ea73 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:53.024339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d78b2c-f4a3-49ce-a497-e6d09c51dc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:53.024926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d78b2c-f4a3-49ce-a497-e6d09c51dc01 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:53.025202 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d78b2c-f4a3-49ce-a497-e6d09c51dc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:53.025467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d78b2c-f4a3-49ce-a497-e6d09c51dc01 admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:53.026123 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d78b2c-f4a3-49ce-a497-e6d09c51dc01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:53.027995 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d78b2c-f4a3-49ce-a497-e6d09c51dc01 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:53.157686 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1401/2841] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:04:53 2022] POST /identity/v3/projects => generated 371 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:04:53.161905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0684df-da69-4cc8-907d-3a821069fe06 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:53.172421 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0684df-da69-4cc8-907d-3a821069fe06 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:53.236220 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fc715-d9f7-48ff-8fd3-bf4b8aa4645c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:53.236818 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc715-d9f7-48ff-8fd3-bf4b8aa4645c admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:53.237103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc715-d9f7-48ff-8fd3-bf4b8aa4645c admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:53.237357 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92fc715-d9f7-48ff-8fd3-bf4b8aa4645c admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:53.238011 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fc715-d9f7-48ff-8fd3-bf4b8aa4645c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:53.239919 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92fc715-d9f7-48ff-8fd3-bf4b8aa4645c admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:53.346525 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1441/2842] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:04:53 2022] POST /identity/v3/users => generated 286 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:04:53.350596 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d78b2c-f4a3-49ce-a497-e6d09c51dc01 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:53.361174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d78b2c-f4a3-49ce-a497-e6d09c51dc01 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:53.365645 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132cf236-5052-44bd-8648-c79e5c84cf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:53.366265 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cf236-5052-44bd-8648-c79e5c84cf1b admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:53.366543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cf236-5052-44bd-8648-c79e5c84cf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:53.366809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132cf236-5052-44bd-8648-c79e5c84cf1b admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:53.367459 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132cf236-5052-44bd-8648-c79e5c84cf1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:53.369026 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132cf236-5052-44bd-8648-c79e5c84cf1b admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:53.381884 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1402/2843] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:04:53.383236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fc715-d9f7-48ff-8fd3-bf4b8aa4645c admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:53.398974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92fc715-d9f7-48ff-8fd3-bf4b8aa4645c admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:53.407564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cb1704-5863-4daf-a8dd-7bee19fb4a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:53.407564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb1704-5863-4daf-a8dd-7bee19fb4a87 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:53.407564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb1704-5863-4daf-a8dd-7bee19fb4a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:53.407564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cb1704-5863-4daf-a8dd-7bee19fb4a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:53.407978 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb1704-5863-4daf-a8dd-7bee19fb4a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:53.408243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cb1704-5863-4daf-a8dd-7bee19fb4a87 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:53.419288 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1442/2844] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:04:53.423539 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132cf236-5052-44bd-8648-c79e5c84cf1b admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:53.433446 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132cf236-5052-44bd-8648-c79e5c84cf1b admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:53.438216 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d6e93-4a42-449a-a6f1-4206b56c50c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:53.460757 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d6e93-4a42-449a-a6f1-4206b56c50c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:53.460757 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d6e93-4a42-449a-a6f1-4206b56c50c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:53.460757 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d6e93-4a42-449a-a6f1-4206b56c50c2 admin admin] PATH_INFO: `/v3/projects/b9d4ae2217ce4378aca743ebba32e7d6/users/e763756287b04471bbc6b4f8afd28b8b/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:53.460757 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d6e93-4a42-449a-a6f1-4206b56c50c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d4ae2217ce4378aca743ebba32e7d6, user_id=e763756287b04471bbc6b4f8afd28b8b, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:53.460757 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d6e93-4a42-449a-a6f1-4206b56c50c2 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:53.536441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1403/2845] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:04:53 2022] PUT /identity/v3/projects/b9d4ae2217ce4378aca743ebba32e7d6/users/e763756287b04471bbc6b4f8afd28b8b/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:04:53.541511 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e5beb-4ffd-4c69-afcb-5ab55e6f95ce None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:53.541872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e5beb-4ffd-4c69-afcb-5ab55e6f95ce None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:53.542150 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18e5beb-4ffd-4c69-afcb-5ab55e6f95ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:53.561189 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c18e5beb-4ffd-4c69-afcb-5ab55e6f95ce None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:53.562425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c18e5beb-4ffd-4c69-afcb-5ab55e6f95ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:53.620776 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1443/2846] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:04:53.630600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d6e93-4a42-449a-a6f1-4206b56c50c2 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:53.632410 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-148d6e93-4a42-449a-a6f1-4206b56c50c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:53.690864 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d6e93-4a42-449a-a6f1-4206b56c50c2 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:53.694526 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9fbb8-8c5f-4ccc-9900-9648aa154958 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:53.695140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9fbb8-8c5f-4ccc-9900-9648aa154958 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:53.695420 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9fbb8-8c5f-4ccc-9900-9648aa154958 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:53.695690 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9fbb8-8c5f-4ccc-9900-9648aa154958 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:53.705260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9fbb8-8c5f-4ccc-9900-9648aa154958 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:53.706994 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9fbb8-8c5f-4ccc-9900-9648aa154958 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:53.716787 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1404/2847] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:04:55.056282 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371592cd-f09b-40d6-b208-dca16aef253a None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:55.061713 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371592cd-f09b-40d6-b208-dca16aef253a None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:55.061713 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371592cd-f09b-40d6-b208-dca16aef253a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:55.080304 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:04:55.089173 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-371592cd-f09b-40d6-b208-dca16aef253a None None] MFA Rules not processed for user `e763756287b04471bbc6b4f8afd28b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:55.090594 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-371592cd-f09b-40d6-b208-dca16aef253a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:55.161185 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1444/2848] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:04:55.172276 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9fbb8-8c5f-4ccc-9900-9648aa154958 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:04:55.173725 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-f6d9fbb8-8c5f-4ccc-9900-9648aa154958 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:55.234644 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9fbb8-8c5f-4ccc-9900-9648aa154958 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:04:55.238323 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20793f31-2c67-4504-a0cf-2925b9455c7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:04:55.238924 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20793f31-2c67-4504-a0cf-2925b9455c7f service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:55.239203 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20793f31-2c67-4504-a0cf-2925b9455c7f service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:55.239472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20793f31-2c67-4504-a0cf-2925b9455c7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:55.247865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20793f31-2c67-4504-a0cf-2925b9455c7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:04:55.249961 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20793f31-2c67-4504-a0cf-2925b9455c7f service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:04:55.260833 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1405/2849] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:04:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:04:57.088728 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0d07d-b062-480a-8c6e-5714e284e194 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:04:57.089448 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0d07d-b062-480a-8c6e-5714e284e194 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:04:57.089448 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c0d07d-b062-480a-8c6e-5714e284e194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:04:57.109507 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-56c0d07d-b062-480a-8c6e-5714e284e194 None None] MFA Rules not processed for user `e763756287b04471bbc6b4f8afd28b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:04:57.110757 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-56c0d07d-b062-480a-8c6e-5714e284e194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:04:57.117481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1445/2850] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:04:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:05:05.258868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=78,keystone:INSERT=2 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:05:07.102827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=30 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:05:17.643339 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d83f3f-3f36-4174-a2ba-69d59086b0c2 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:17.643868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d83f3f-3f36-4174-a2ba-69d59086b0c2 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:17.643868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d83f3f-3f36-4174-a2ba-69d59086b0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:17.662846 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-e7d83f3f-3f36-4174-a2ba-69d59086b0c2 None None] MFA Rules not processed for user `e763756287b04471bbc6b4f8afd28b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:05:17.664092 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e7d83f3f-3f36-4174-a2ba-69d59086b0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:17.670938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1406/2851] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:05:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:05:22.234826 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0d07d-b062-480a-8c6e-5714e284e194 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:22.236323 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-56c0d07d-b062-480a-8c6e-5714e284e194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:22.250219 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c0d07d-b062-480a-8c6e-5714e284e194 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:22.254825 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071ae97-610b-485f-9208-e2d38d98a0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:22.255583 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071ae97-610b-485f-9208-e2d38d98a0ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:22.255723 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071ae97-610b-485f-9208-e2d38d98a0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:22.256030 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4071ae97-610b-485f-9208-e2d38d98a0ac admin admin] PATH_INFO: `/v3/users/48dfaf9f066045c2b80f4f13a0582f6e` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:22.257144 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071ae97-610b-485f-9208-e2d38d98a0ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=48dfaf9f066045c2b80f4f13a0582f6e)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:22.259117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4071ae97-610b-485f-9208-e2d38d98a0ac admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:22.449394 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4071ae97-610b-485f-9208-e2d38d98a0ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48dfaf9f066045c2b80f4f13a0582f6e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:22.527322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4071ae97-610b-485f-9208-e2d38d98a0ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48dfaf9f066045c2b80f4f13a0582f6e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:22.542780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4071ae97-610b-485f-9208-e2d38d98a0ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48dfaf9f066045c2b80f4f13a0582f6e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:22.543493 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4071ae97-610b-485f-9208-e2d38d98a0ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48dfaf9f066045c2b80f4f13a0582f6e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:22.544159 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-4071ae97-610b-485f-9208-e2d38d98a0ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48dfaf9f066045c2b80f4f13a0582f6e'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:22.561172 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1446/2852] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:05:22 2022] DELETE /identity/v3/users/48dfaf9f066045c2b80f4f13a0582f6e => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:05:22.563185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5df2a0-860b-412a-baaf-32971378418c None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:22.563999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5df2a0-860b-412a-baaf-32971378418c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:22.566110 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5df2a0-860b-412a-baaf-32971378418c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:22.583613 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-fc5df2a0-860b-412a-baaf-32971378418c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:05:22.588989 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-fc5df2a0-860b-412a-baaf-32971378418c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:22.648432 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1407/2853] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:05:22.658336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071ae97-610b-485f-9208-e2d38d98a0ac admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:22.659773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-4071ae97-610b-485f-9208-e2d38d98a0ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:22.724025 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4071ae97-610b-485f-9208-e2d38d98a0ac admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:22.728317 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c51597-cee0-4eba-b9b2-fc47e835bf72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:22.729037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c51597-cee0-4eba-b9b2-fc47e835bf72 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:22.729410 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c51597-cee0-4eba-b9b2-fc47e835bf72 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:22.730243 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c51597-cee0-4eba-b9b2-fc47e835bf72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:22.740480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c51597-cee0-4eba-b9b2-fc47e835bf72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:22.741991 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c51597-cee0-4eba-b9b2-fc47e835bf72 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:22.752396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1447/2854] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:05:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:05:23.245009 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5df2a0-860b-412a-baaf-32971378418c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:23.246496 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-fc5df2a0-860b-412a-baaf-32971378418c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:23.259497 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5df2a0-860b-412a-baaf-32971378418c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:23.269055 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a6532-19df-4204-a4e3-e34732e721ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:23.269707 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a6532-19df-4204-a4e3-e34732e721ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:23.270042 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a6532-19df-4204-a4e3-e34732e721ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:23.270315 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a6532-19df-4204-a4e3-e34732e721ec admin admin] PATH_INFO: `/v3/projects/0f467720ddbc4fb08524573719b2e7ad` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:23.271583 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a6532-19df-4204-a4e3-e34732e721ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f467720ddbc4fb08524573719b2e7ad)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:23.280876 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a6532-19df-4204-a4e3-e34732e721ec admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:23.392917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-109a6532-19df-4204-a4e3-e34732e721ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f467720ddbc4fb08524573719b2e7ad', 'request_id': 'req-109a6532-19df-4204-a4e3-e34732e721ec'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:23.398843 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-109a6532-19df-4204-a4e3-e34732e721ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f467720ddbc4fb08524573719b2e7ad', 'request_id': 'req-109a6532-19df-4204-a4e3-e34732e721ec'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:23.405002 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-109a6532-19df-4204-a4e3-e34732e721ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f467720ddbc4fb08524573719b2e7ad', 'request_id': 'req-109a6532-19df-4204-a4e3-e34732e721ec'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:23.546773 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-109a6532-19df-4204-a4e3-e34732e721ec admin admin] The token cache is being invalidate because project 0f467720ddbc4fb08524573719b2e7ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:05:23.547074 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-109a6532-19df-4204-a4e3-e34732e721ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:23.547737 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-109a6532-19df-4204-a4e3-e34732e721ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:23.549221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1408/2855] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:05:23 2022] DELETE /identity/v3/projects/0f467720ddbc4fb08524573719b2e7ad => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:05:23.554130 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370980e-0420-4a1d-8779-1aa699096d1c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:23.554617 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370980e-0420-4a1d-8779-1aa699096d1c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:23.554730 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370980e-0420-4a1d-8779-1aa699096d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:23.575230 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-7370980e-0420-4a1d-8779-1aa699096d1c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:05:23.576443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7370980e-0420-4a1d-8779-1aa699096d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:23.634850 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1448/2856] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:05:23.642675 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85c129-b7c4-4630-9b1d-331ec541576c None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:23.643195 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85c129-b7c4-4630-9b1d-331ec541576c None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:23.643313 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d85c129-b7c4-4630-9b1d-331ec541576c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:23.662542 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-9d85c129-b7c4-4630-9b1d-331ec541576c None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:05:23.663740 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-9d85c129-b7c4-4630-9b1d-331ec541576c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:23.672107 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1409/2857] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:05:23.676129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370980e-0420-4a1d-8779-1aa699096d1c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:23.685596 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370980e-0420-4a1d-8779-1aa699096d1c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:23.690895 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deff40b-8123-4afd-8b1b-ca1e66160ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:23.691483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deff40b-8123-4afd-8b1b-ca1e66160ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:23.691762 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deff40b-8123-4afd-8b1b-ca1e66160ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:23.692034 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deff40b-8123-4afd-8b1b-ca1e66160ba5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:23.692669 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deff40b-8123-4afd-8b1b-ca1e66160ba5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93427) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:05:23.692974 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deff40b-8123-4afd-8b1b-ca1e66160ba5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:23.694614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deff40b-8123-4afd-8b1b-ca1e66160ba5 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:23.706113 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1449/2858] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:05:23.710528 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85c129-b7c4-4630-9b1d-331ec541576c None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:23.720708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d85c129-b7c4-4630-9b1d-331ec541576c None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:23.725608 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa43d2-47c1-4315-a05d-5a400ca5fbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:23.726252 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa43d2-47c1-4315-a05d-5a400ca5fbad admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:23.726535 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa43d2-47c1-4315-a05d-5a400ca5fbad admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:23.726805 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa43d2-47c1-4315-a05d-5a400ca5fbad admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:23.727441 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa43d2-47c1-4315-a05d-5a400ca5fbad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:23.729375 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afa43d2-47c1-4315-a05d-5a400ca5fbad admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:23.825402 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1410/2859] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:05:23 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:05:23.829686 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deff40b-8123-4afd-8b1b-ca1e66160ba5 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:23.834776 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:05:23.840470 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deff40b-8123-4afd-8b1b-ca1e66160ba5 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:23.897615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0cfba-cadb-44fb-98e2-07c7d569e79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:23.897615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0cfba-cadb-44fb-98e2-07c7d569e79e admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:23.897615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0cfba-cadb-44fb-98e2-07c7d569e79e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:23.897615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0cfba-cadb-44fb-98e2-07c7d569e79e admin admin] PATH_INFO: `/v3/users` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:23.898158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0cfba-cadb-44fb-98e2-07c7d569e79e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:23.900132 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f0cfba-cadb-44fb-98e2-07c7d569e79e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:23.981560 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1450/2860] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:05:23 2022] POST /identity/v3/users => generated 288 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:05:23.983819 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa43d2-47c1-4315-a05d-5a400ca5fbad admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:23.994758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa43d2-47c1-4315-a05d-5a400ca5fbad admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:23.999698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9486f1a-6e14-4e0a-aad2-14ad65878859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:24.000327 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9486f1a-6e14-4e0a-aad2-14ad65878859 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:24.000602 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9486f1a-6e14-4e0a-aad2-14ad65878859 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:24.000870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9486f1a-6e14-4e0a-aad2-14ad65878859 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:24.001500 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9486f1a-6e14-4e0a-aad2-14ad65878859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:24.003365 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9486f1a-6e14-4e0a-aad2-14ad65878859 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:24.014286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1411/2861] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:05:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:05:24.018759 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0cfba-cadb-44fb-98e2-07c7d569e79e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:24.029053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f0cfba-cadb-44fb-98e2-07c7d569e79e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:24.034957 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02283e4-da4f-442b-b117-85ef87ab880b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:24.035554 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02283e4-da4f-442b-b117-85ef87ab880b admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:24.035833 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02283e4-da4f-442b-b117-85ef87ab880b admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:24.036105 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02283e4-da4f-442b-b117-85ef87ab880b admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:24.036737 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02283e4-da4f-442b-b117-85ef87ab880b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:24.038379 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02283e4-da4f-442b-b117-85ef87ab880b admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:24.049690 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1451/2862] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:05:24.054124 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9486f1a-6e14-4e0a-aad2-14ad65878859 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:24.064400 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9486f1a-6e14-4e0a-aad2-14ad65878859 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:24.069050 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce89e0-a3ee-4e29-b268-f32765ff9c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:24.069698 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce89e0-a3ee-4e29-b268-f32765ff9c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:24.069998 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce89e0-a3ee-4e29-b268-f32765ff9c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:24.070268 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce89e0-a3ee-4e29-b268-f32765ff9c36 admin admin] PATH_INFO: `/v3/projects/327e2fb394ee4ebb882480bf59965576/users/61f042c6fe40457ea80e0a107adedbd3/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:24.085382 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce89e0-a3ee-4e29-b268-f32765ff9c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=327e2fb394ee4ebb882480bf59965576, user_id=61f042c6fe40457ea80e0a107adedbd3, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:24.088086 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce89e0-a3ee-4e29-b268-f32765ff9c36 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:24.186905 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1412/2863] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:05:24 2022] PUT /identity/v3/projects/327e2fb394ee4ebb882480bf59965576/users/61f042c6fe40457ea80e0a107adedbd3/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:05:24.206080 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f816c-a8a1-4764-9d0c-f846470f680d None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:24.206360 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f816c-a8a1-4764-9d0c-f846470f680d None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:24.206625 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f816c-a8a1-4764-9d0c-f846470f680d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:24.235685 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-2f9f816c-a8a1-4764-9d0c-f846470f680d None None] MFA Rules not processed for user `61f042c6fe40457ea80e0a107adedbd3`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:05:24.239026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2f9f816c-a8a1-4764-9d0c-f846470f680d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:24.301614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1452/2864] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:05:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:05:24.312436 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce89e0-a3ee-4e29-b268-f32765ff9c36 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:24.314188 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-40ce89e0-a3ee-4e29-b268-f32765ff9c36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:24.328977 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:05:24.372180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce89e0-a3ee-4e29-b268-f32765ff9c36 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:24.375869 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfd729-bda8-4851-b99d-e74a13587299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:24.376467 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfd729-bda8-4851-b99d-e74a13587299 service cinder] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:24.376752 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfd729-bda8-4851-b99d-e74a13587299 service cinder] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:24.377041 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfd729-bda8-4851-b99d-e74a13587299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:24.385286 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccfd729-bda8-4851-b99d-e74a13587299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:24.386735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccfd729-bda8-4851-b99d-e74a13587299 service cinder] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:24.395999 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1413/2865] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:05:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:05:26.294427 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0d8d7-3bad-40b6-a5e6-4e9b8563035f None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:26.294934 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0d8d7-3bad-40b6-a5e6-4e9b8563035f None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:26.295197 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0d8d7-3bad-40b6-a5e6-4e9b8563035f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:26.316195 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-07d0d8d7-3bad-40b6-a5e6-4e9b8563035f None None] MFA Rules not processed for user `61f042c6fe40457ea80e0a107adedbd3`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:05:26.317447 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-07d0d8d7-3bad-40b6-a5e6-4e9b8563035f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:26.328865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1453/2866] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:05:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:05:34.448839 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=20 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:05:36.309658 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=68,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:05:39.651383 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfd729-bda8-4851-b99d-e74a13587299 service cinder] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:39.652778 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8ccfd729-bda8-4851-b99d-e74a13587299 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:39.708938 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfd729-bda8-4851-b99d-e74a13587299 service cinder] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:39.712664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e741b39-cd35-4aee-90c0-851e4fd15ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:39.713260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741b39-cd35-4aee-90c0-851e4fd15ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:39.713793 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741b39-cd35-4aee-90c0-851e4fd15ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:39.714088 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e741b39-cd35-4aee-90c0-851e4fd15ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:39.715814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2e741b39-cd35-4aee-90c0-851e4fd15ef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:39.784167 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e741b39-cd35-4aee-90c0-851e4fd15ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:39.786180 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e741b39-cd35-4aee-90c0-851e4fd15ef5 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:39.796940 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1414/2867] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:05:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:05:42.604214 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823af27-62ec-4857-af44-0287cf9c1214 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:42.604668 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823af27-62ec-4857-af44-0287cf9c1214 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:42.604773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823af27-62ec-4857-af44-0287cf9c1214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:42.624535 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-1823af27-62ec-4857-af44-0287cf9c1214 None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:05:42.625718 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1823af27-62ec-4857-af44-0287cf9c1214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:42.682773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1454/2868] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 20:05:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:05:42.796514 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e741b39-cd35-4aee-90c0-851e4fd15ef5 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:42.798047 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2e741b39-cd35-4aee-90c0-851e4fd15ef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:42.869489 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e741b39-cd35-4aee-90c0-851e4fd15ef5 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:42.874862 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e767e7-a81c-4332-ae6f-ced23c91b9db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:42.875461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e767e7-a81c-4332-ae6f-ced23c91b9db service swift] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:42.875734 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e767e7-a81c-4332-ae6f-ced23c91b9db service swift] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:42.876000 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e767e7-a81c-4332-ae6f-ced23c91b9db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:42.884138 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e767e7-a81c-4332-ae6f-ced23c91b9db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:42.888633 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e767e7-a81c-4332-ae6f-ced23c91b9db service swift] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:42.894716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1415/2869] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 20:05:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 20:05:43.955400 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823af27-62ec-4857-af44-0287cf9c1214 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:43.957827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1823af27-62ec-4857-af44-0287cf9c1214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:43.971517 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823af27-62ec-4857-af44-0287cf9c1214 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:43.975960 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524984c-3f05-4b9b-b62d-1e62a803e78d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:43.976886 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524984c-3f05-4b9b-b62d-1e62a803e78d service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:43.977245 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524984c-3f05-4b9b-b62d-1e62a803e78d service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:43.977522 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524984c-3f05-4b9b-b62d-1e62a803e78d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:43.979405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-2524984c-3f05-4b9b-b62d-1e62a803e78d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:44.001720 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524984c-3f05-4b9b-b62d-1e62a803e78d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:44.004381 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524984c-3f05-4b9b-b62d-1e62a803e78d service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:44.023398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1455/2870] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:05:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:05:49.178308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88071b-d4a8-4aac-a6db-519225c4b533 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:49.179029 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88071b-d4a8-4aac-a6db-519225c4b533 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:49.179029 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88071b-d4a8-4aac-a6db-519225c4b533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:49.201397 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-2a88071b-d4a8-4aac-a6db-519225c4b533 None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:05:49.202710 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2a88071b-d4a8-4aac-a6db-519225c4b533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:49.211366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1416/2871] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 20:05:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:05:49.229735 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524984c-3f05-4b9b-b62d-1e62a803e78d service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:49.245299 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524984c-3f05-4b9b-b62d-1e62a803e78d service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:49.251217 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f85c23-6b44-4df3-9743-73ec27806ba6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:49.251831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f85c23-6b44-4df3-9743-73ec27806ba6 service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:49.252121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f85c23-6b44-4df3-9743-73ec27806ba6 service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:49.252388 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f85c23-6b44-4df3-9743-73ec27806ba6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:49.262801 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f85c23-6b44-4df3-9743-73ec27806ba6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:49.264595 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f85c23-6b44-4df3-9743-73ec27806ba6 service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:49.275537 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1456/2872] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 20:05:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 20:05:50.318153 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85209d0f-f5e1-40c4-bdd3-128e3fda8ac1 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:50.318669 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85209d0f-f5e1-40c4-bdd3-128e3fda8ac1 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:50.318910 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85209d0f-f5e1-40c4-bdd3-128e3fda8ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:50.340643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-85209d0f-f5e1-40c4-bdd3-128e3fda8ac1 None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:05:50.343560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-85209d0f-f5e1-40c4-bdd3-128e3fda8ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:50.351871 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1417/2873] 10.222.0.42 () {58 vars in 967 bytes} [Wed Sep 14 20:05:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:05:50.368327 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f85c23-6b44-4df3-9743-73ec27806ba6 service swift] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:50.381341 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f85c23-6b44-4df3-9743-73ec27806ba6 service swift] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:50.388648 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1938e-5b2e-4cee-96be-cc795d509ba2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:50.389552 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1938e-5b2e-4cee-96be-cc795d509ba2 service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:50.390038 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1938e-5b2e-4cee-96be-cc795d509ba2 service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:50.390484 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1938e-5b2e-4cee-96be-cc795d509ba2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:50.400621 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1938e-5b2e-4cee-96be-cc795d509ba2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:50.402679 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f1938e-5b2e-4cee-96be-cc795d509ba2 service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:50.412997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1457/2874] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 20:05:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 20:05:51.104756 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ef934-7c9e-42e5-a427-430968e16d76 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:51.105046 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ef934-7c9e-42e5-a427-430968e16d76 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:51.105324 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ef934-7c9e-42e5-a427-430968e16d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:51.127479 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-0c3ef934-7c9e-42e5-a427-430968e16d76 None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:05:51.129642 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0c3ef934-7c9e-42e5-a427-430968e16d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:51.137083 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1418/2875] 10.222.0.42 () {58 vars in 1009 bytes} [Wed Sep 14 20:05:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:05:51.151865 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1938e-5b2e-4cee-96be-cc795d509ba2 service swift] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:51.162740 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1938e-5b2e-4cee-96be-cc795d509ba2 service swift] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:51.167731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1714006-d6ed-4fb1-9eef-ed05ca0a3bb2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:51.168322 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1714006-d6ed-4fb1-9eef-ed05ca0a3bb2 service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:51.168762 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1714006-d6ed-4fb1-9eef-ed05ca0a3bb2 service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:51.169251 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1714006-d6ed-4fb1-9eef-ed05ca0a3bb2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:51.178573 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1714006-d6ed-4fb1-9eef-ed05ca0a3bb2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:51.180121 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1714006-d6ed-4fb1-9eef-ed05ca0a3bb2 service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:51.189037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1458/2876] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 20:05:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 20:05:51.830130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fbfa4-6a04-449b-801d-f0e4f78cf56d None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:51.830899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fbfa4-6a04-449b-801d-f0e4f78cf56d None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:51.830899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fbfa4-6a04-449b-801d-f0e4f78cf56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:51.853161 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-ab9fbfa4-6a04-449b-801d-f0e4f78cf56d None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:05:51.855249 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-ab9fbfa4-6a04-449b-801d-f0e4f78cf56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:51.865356 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1419/2877] 10.222.0.42 () {58 vars in 967 bytes} [Wed Sep 14 20:05:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:05:51.879006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1714006-d6ed-4fb1-9eef-ed05ca0a3bb2 service swift] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:51.890288 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1714006-d6ed-4fb1-9eef-ed05ca0a3bb2 service swift] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:51.896782 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adaabce-1061-4d4a-b6fa-997246de19b2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:51.897429 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaabce-1061-4d4a-b6fa-997246de19b2 service swift] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:51.897711 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaabce-1061-4d4a-b6fa-997246de19b2 service swift] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:51.898041 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adaabce-1061-4d4a-b6fa-997246de19b2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:51.907327 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adaabce-1061-4d4a-b6fa-997246de19b2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:51.908964 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adaabce-1061-4d4a-b6fa-997246de19b2 service swift] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:51.918931 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1459/2878] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 20:05:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 20:05:55.310148 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fbfa4-6a04-449b-801d-f0e4f78cf56d None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:55.320118 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:05:55.395482 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fbfa4-6a04-449b-801d-f0e4f78cf56d None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:55.400427 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d465-daad-4712-838f-408fa18b21c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:55.401275 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d465-daad-4712-838f-408fa18b21c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:55.401729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d465-daad-4712-838f-408fa18b21c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:55.402141 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4884d465-daad-4712-838f-408fa18b21c9 admin admin] PATH_INFO: `/v3/users/61f042c6fe40457ea80e0a107adedbd3` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:55.403305 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884d465-daad-4712-838f-408fa18b21c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61f042c6fe40457ea80e0a107adedbd3)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:55.405231 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4884d465-daad-4712-838f-408fa18b21c9 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:55.602623 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4884d465-daad-4712-838f-408fa18b21c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61f042c6fe40457ea80e0a107adedbd3'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:55.621269 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4884d465-daad-4712-838f-408fa18b21c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61f042c6fe40457ea80e0a107adedbd3'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:55.695193 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4884d465-daad-4712-838f-408fa18b21c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61f042c6fe40457ea80e0a107adedbd3'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:55.695821 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4884d465-daad-4712-838f-408fa18b21c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61f042c6fe40457ea80e0a107adedbd3'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:55.709631 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-4884d465-daad-4712-838f-408fa18b21c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61f042c6fe40457ea80e0a107adedbd3'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:55.716524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1420/2879] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:05:55 2022] DELETE /identity/v3/users/61f042c6fe40457ea80e0a107adedbd3 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:05:55.721259 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf6756-ff81-4472-9b71-4ea6496c0bf6 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:55.723221 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf6756-ff81-4472-9b71-4ea6496c0bf6 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:55.723580 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bf6756-ff81-4472-9b71-4ea6496c0bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:55.745745 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-44bf6756-ff81-4472-9b71-4ea6496c0bf6 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:05:55.747087 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-44bf6756-ff81-4472-9b71-4ea6496c0bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:55.820853 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1460/2880] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:05:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:05:55.830865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d465-daad-4712-838f-408fa18b21c9 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:55.832865 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4884d465-daad-4712-838f-408fa18b21c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:55.914809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4884d465-daad-4712-838f-408fa18b21c9 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:55.919195 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61484003-e912-48fe-bd7c-fec2710d664b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:55.920059 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61484003-e912-48fe-bd7c-fec2710d664b service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:55.920341 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61484003-e912-48fe-bd7c-fec2710d664b service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:55.921891 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61484003-e912-48fe-bd7c-fec2710d664b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:55.932023 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61484003-e912-48fe-bd7c-fec2710d664b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:55.933456 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61484003-e912-48fe-bd7c-fec2710d664b service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:55.943543 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1421/2881] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:05:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:05:56.263761 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf6756-ff81-4472-9b71-4ea6496c0bf6 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:56.270123 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-44bf6756-ff81-4472-9b71-4ea6496c0bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:56.279508 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bf6756-ff81-4472-9b71-4ea6496c0bf6 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:56.286175 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adb6fd-db4e-4b80-baca-fde28a723543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:56.287260 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adb6fd-db4e-4b80-baca-fde28a723543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:56.287949 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adb6fd-db4e-4b80-baca-fde28a723543 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:56.288405 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adb6fd-db4e-4b80-baca-fde28a723543 admin admin] PATH_INFO: `/v3/projects/327e2fb394ee4ebb882480bf59965576` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:56.289887 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adb6fd-db4e-4b80-baca-fde28a723543 admin admin] RBAC: Authorizing `identity:delete_project(project_id=327e2fb394ee4ebb882480bf59965576)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:56.291888 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adb6fd-db4e-4b80-baca-fde28a723543 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:56.391244 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:05:56.402425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-90adb6fd-db4e-4b80-baca-fde28a723543 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '327e2fb394ee4ebb882480bf59965576', 'request_id': 'req-90adb6fd-db4e-4b80-baca-fde28a723543'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:56.492814 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-90adb6fd-db4e-4b80-baca-fde28a723543 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '327e2fb394ee4ebb882480bf59965576', 'request_id': 'req-90adb6fd-db4e-4b80-baca-fde28a723543'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:56.501318 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-90adb6fd-db4e-4b80-baca-fde28a723543 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '327e2fb394ee4ebb882480bf59965576', 'request_id': 'req-90adb6fd-db4e-4b80-baca-fde28a723543'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:56.515293 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-90adb6fd-db4e-4b80-baca-fde28a723543 admin admin] The token cache is being invalidate because project 327e2fb394ee4ebb882480bf59965576 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:05:56.515533 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-90adb6fd-db4e-4b80-baca-fde28a723543 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:56.516294 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-90adb6fd-db4e-4b80-baca-fde28a723543 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:05:56.518440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1461/2882] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:05:56 2022] DELETE /identity/v3/projects/327e2fb394ee4ebb882480bf59965576 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:05:57.043715 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61484003-e912-48fe-bd7c-fec2710d664b service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:05:57.045306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-61484003-e912-48fe-bd7c-fec2710d664b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:57.060318 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61484003-e912-48fe-bd7c-fec2710d664b service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:05:57.097396 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:05:57.135016 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a9af9-7716-4a06-ba63-0e254224c922 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:05:57.135915 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a9af9-7716-4a06-ba63-0e254224c922 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:05:57.136253 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a9af9-7716-4a06-ba63-0e254224c922 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:05:57.136526 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a9af9-7716-4a06-ba63-0e254224c922 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:05:57.138297 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2e8a9af9-7716-4a06-ba63-0e254224c922 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:05:57.224036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a9af9-7716-4a06-ba63-0e254224c922 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:05:57.226313 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a9af9-7716-4a06-ba63-0e254224c922 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:05:57.242709 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1422/2883] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:05:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:06:00.097528 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a0341-30fd-4e2f-9087-9776b3677bf5 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:00.098076 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a0341-30fd-4e2f-9087-9776b3677bf5 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:00.098524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a0341-30fd-4e2f-9087-9776b3677bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:00.150709 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-442a0341-30fd-4e2f-9087-9776b3677bf5 None None] MFA Rules not processed for user `7d5605f645a74b1b842733dda256193c`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:06:00.152158 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-442a0341-30fd-4e2f-9087-9776b3677bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:00.225954 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1462/2884] 10.222.0.42 () {58 vars in 967 bytes} [Wed Sep 14 20:06:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:06:00.240930 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a9af9-7716-4a06-ba63-0e254224c922 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:00.242880 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-2e8a9af9-7716-4a06-ba63-0e254224c922 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:00.343290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a9af9-7716-4a06-ba63-0e254224c922 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:00.349160 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf99242-ea7c-46da-825a-1d8c6d8ba070 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3d2f1ab1bfd435183e33168400c0b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:00.349754 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf99242-ea7c-46da-825a-1d8c6d8ba070 service swift] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:00.350069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf99242-ea7c-46da-825a-1d8c6d8ba070 service swift] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:00.350337 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf99242-ea7c-46da-825a-1d8c6d8ba070 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:00.359075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf99242-ea7c-46da-825a-1d8c6d8ba070 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:00.360655 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf99242-ea7c-46da-825a-1d8c6d8ba070 service swift] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:00.369368 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1423/2885] 10.222.0.42 () {60 vars in 1384 bytes} [Wed Sep 14 20:06:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 20:06:03.450366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a0341-30fd-4e2f-9087-9776b3677bf5 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:03.451872 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-442a0341-30fd-4e2f-9087-9776b3677bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:03.520749 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a0341-30fd-4e2f-9087-9776b3677bf5 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:03.525433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306180c7-f615-4a01-9f7e-029c49085d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:03.528861 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306180c7-f615-4a01-9f7e-029c49085d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:03.528861 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306180c7-f615-4a01-9f7e-029c49085d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:03.528861 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306180c7-f615-4a01-9f7e-029c49085d52 admin admin] PATH_INFO: `/v3/users/e763756287b04471bbc6b4f8afd28b8b` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:03.528861 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306180c7-f615-4a01-9f7e-029c49085d52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e763756287b04471bbc6b4f8afd28b8b)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:03.529642 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306180c7-f615-4a01-9f7e-029c49085d52 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:03.819653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:06:03.828754 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-306180c7-f615-4a01-9f7e-029c49085d52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e763756287b04471bbc6b4f8afd28b8b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:03.926512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf99242-ea7c-46da-825a-1d8c6d8ba070 service swift] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:03.927103 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-306180c7-f615-4a01-9f7e-029c49085d52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e763756287b04471bbc6b4f8afd28b8b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:03.932140 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-adf99242-ea7c-46da-825a-1d8c6d8ba070 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:03.940220 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf99242-ea7c-46da-825a-1d8c6d8ba070 service swift] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:03.940905 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-306180c7-f615-4a01-9f7e-029c49085d52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e763756287b04471bbc6b4f8afd28b8b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:03.942318 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-306180c7-f615-4a01-9f7e-029c49085d52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e763756287b04471bbc6b4f8afd28b8b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:03.943015 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-306180c7-f615-4a01-9f7e-029c49085d52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e763756287b04471bbc6b4f8afd28b8b'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:03.944566 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d2b566-bc30-4ae3-bc9c-c85171b07024 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:03.945187 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2b566-bc30-4ae3-bc9c-c85171b07024 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:03.945461 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2b566-bc30-4ae3-bc9c-c85171b07024 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:03.945729 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d2b566-bc30-4ae3-bc9c-c85171b07024 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:03.947477 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-e9d2b566-bc30-4ae3-bc9c-c85171b07024 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:03.955765 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1463/2886] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:06:03 2022] DELETE /identity/v3/users/e763756287b04471bbc6b4f8afd28b8b => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:06:03.960889 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d2b566-bc30-4ae3-bc9c-c85171b07024 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:03.962145 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d2b566-bc30-4ae3-bc9c-c85171b07024 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:03.973146 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1424/2887] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:06:04.265023 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306180c7-f615-4a01-9f7e-029c49085d52 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:04.266605 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-306180c7-f615-4a01-9f7e-029c49085d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:04.351663 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306180c7-f615-4a01-9f7e-029c49085d52 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:04.356359 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a9cb62-bc76-49d8-bc5e-724ff1455582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:04.357026 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9cb62-bc76-49d8-bc5e-724ff1455582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:04.357302 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9cb62-bc76-49d8-bc5e-724ff1455582 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:04.357571 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a9cb62-bc76-49d8-bc5e-724ff1455582 admin admin] PATH_INFO: `/v3/projects/b9d4ae2217ce4378aca743ebba32e7d6` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:04.358717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9cb62-bc76-49d8-bc5e-724ff1455582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9d4ae2217ce4378aca743ebba32e7d6)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:04.360626 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a9cb62-bc76-49d8-bc5e-724ff1455582 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:04.476699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d1a9cb62-bc76-49d8-bc5e-724ff1455582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9d4ae2217ce4378aca743ebba32e7d6', 'request_id': 'req-d1a9cb62-bc76-49d8-bc5e-724ff1455582'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:04.606084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d1a9cb62-bc76-49d8-bc5e-724ff1455582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9d4ae2217ce4378aca743ebba32e7d6', 'request_id': 'req-d1a9cb62-bc76-49d8-bc5e-724ff1455582'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:04.613002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d1a9cb62-bc76-49d8-bc5e-724ff1455582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9d4ae2217ce4378aca743ebba32e7d6', 'request_id': 'req-d1a9cb62-bc76-49d8-bc5e-724ff1455582'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:04.624478 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d1a9cb62-bc76-49d8-bc5e-724ff1455582 admin admin] The token cache is being invalidate because project b9d4ae2217ce4378aca743ebba32e7d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:06:04.624774 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d1a9cb62-bc76-49d8-bc5e-724ff1455582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:04.625426 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d1a9cb62-bc76-49d8-bc5e-724ff1455582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:04.627117 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1464/2888] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:06:04 2022] DELETE /identity/v3/projects/b9d4ae2217ce4378aca743ebba32e7d6 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:06:04.632178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4a97e-c784-4e22-8882-c439bf6c7533 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:04.632463 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4a97e-c784-4e22-8882-c439bf6c7533 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:04.632741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc4a97e-c784-4e22-8882-c439bf6c7533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:04.652758 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-dcc4a97e-c784-4e22-8882-c439bf6c7533 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:06:04.654130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-dcc4a97e-c784-4e22-8882-c439bf6c7533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:04.688075 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=100 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:06:04.715302 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1425/2889] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:06:04.723660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9cae6-71ab-4811-b9e6-8264cb8932f4 None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:04.723940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9cae6-71ab-4811-b9e6-8264cb8932f4 None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:04.724213 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9cae6-71ab-4811-b9e6-8264cb8932f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:04.744004 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-37d9cae6-71ab-4811-b9e6-8264cb8932f4 None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:06:04.745212 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-37d9cae6-71ab-4811-b9e6-8264cb8932f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:04.753362 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1465/2890] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 20:06:04.757717 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4a97e-c784-4e22-8882-c439bf6c7533 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:04.767557 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc4a97e-c784-4e22-8882-c439bf6c7533 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:04.773099 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78d782-09eb-4d54-bc03-f64a78de6f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:04.773704 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78d782-09eb-4d54-bc03-f64a78de6f10 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:04.774015 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78d782-09eb-4d54-bc03-f64a78de6f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:04.774285 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78d782-09eb-4d54-bc03-f64a78de6f10 admin admin] PATH_INFO: `/v3/domains` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:04.774927 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78d782-09eb-4d54-bc03-f64a78de6f10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93426) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 20:06:04.775229 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78d782-09eb-4d54-bc03-f64a78de6f10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:04.776856 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78d782-09eb-4d54-bc03-f64a78de6f10 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:04.789735 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1426/2891] 10.222.0.42 () {58 vars in 1153 bytes} [Wed Sep 14 20:06:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 20:06:04.794176 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9cae6-71ab-4811-b9e6-8264cb8932f4 None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:04.804002 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9cae6-71ab-4811-b9e6-8264cb8932f4 None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:04.808773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9117fd-c39c-4189-b202-bd35b5c03bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:04.809361 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9117fd-c39c-4189-b202-bd35b5c03bef admin admin] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:04.809633 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9117fd-c39c-4189-b202-bd35b5c03bef admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:04.809949 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9117fd-c39c-4189-b202-bd35b5c03bef admin admin] PATH_INFO: `/v3/projects` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:04.810568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9117fd-c39c-4189-b202-bd35b5c03bef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:04.812471 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9117fd-c39c-4189-b202-bd35b5c03bef admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:04.961647 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1466/2892] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:06:04 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 20:06:04.963135 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78d782-09eb-4d54-bc03-f64a78de6f10 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:04.974200 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78d782-09eb-4d54-bc03-f64a78de6f10 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:05.061987 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5ac9d-7cb3-48d7-9407-7f32e0175e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:05.062726 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5ac9d-7cb3-48d7-9407-7f32e0175e27 admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:05.063069 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5ac9d-7cb3-48d7-9407-7f32e0175e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:05.063452 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5ac9d-7cb3-48d7-9407-7f32e0175e27 admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:05.064166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5ac9d-7cb3-48d7-9407-7f32e0175e27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:05.066718 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5ac9d-7cb3-48d7-9407-7f32e0175e27 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:05.153111 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1427/2893] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:06:04 2022] POST /identity/v3/users => generated 290 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:06:05.154233 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9117fd-c39c-4189-b202-bd35b5c03bef admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:05.164268 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9117fd-c39c-4189-b202-bd35b5c03bef admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:05.168881 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11902e9-31b2-4cf6-9562-0c9d4d14bf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:05.169481 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11902e9-31b2-4cf6-9562-0c9d4d14bf9f admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:05.169755 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11902e9-31b2-4cf6-9562-0c9d4d14bf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:05.170053 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11902e9-31b2-4cf6-9562-0c9d4d14bf9f admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:05.170680 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11902e9-31b2-4cf6-9562-0c9d4d14bf9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:05.172267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11902e9-31b2-4cf6-9562-0c9d4d14bf9f admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:05.182486 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1467/2894] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:06:05.186769 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5ac9d-7cb3-48d7-9407-7f32e0175e27 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:05.197290 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5ac9d-7cb3-48d7-9407-7f32e0175e27 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:05.201905 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767a77ba-3f3c-49cc-83da-c47dec4d613f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:05.202499 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a77ba-3f3c-49cc-83da-c47dec4d613f admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:05.202772 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a77ba-3f3c-49cc-83da-c47dec4d613f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:05.203039 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767a77ba-3f3c-49cc-83da-c47dec4d613f admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:05.203664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a77ba-3f3c-49cc-83da-c47dec4d613f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:05.205308 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767a77ba-3f3c-49cc-83da-c47dec4d613f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:05.217658 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1428/2895] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:06:05.221842 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11902e9-31b2-4cf6-9562-0c9d4d14bf9f admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:05.228579 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:06:05.232396 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11902e9-31b2-4cf6-9562-0c9d4d14bf9f admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:05.237098 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd70fe-4c6f-4c5e-b476-99c710685c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:05.237742 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd70fe-4c6f-4c5e-b476-99c710685c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:05.238579 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd70fe-4c6f-4c5e-b476-99c710685c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:05.238831 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd70fe-4c6f-4c5e-b476-99c710685c96 admin admin] PATH_INFO: `/v3/projects/22037e56af1445c3a88064031d7d4254/users/35e2c522107f4744b8357889a07595eb/roles/efa4ca02b2ff4ad6b14d501d4d79bbea` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:05.255183 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd70fe-4c6f-4c5e-b476-99c710685c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22037e56af1445c3a88064031d7d4254, user_id=35e2c522107f4744b8357889a07595eb, role_id=efa4ca02b2ff4ad6b14d501d4d79bbea)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:05.257848 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd70fe-4c6f-4c5e-b476-99c710685c96 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:05.328909 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1468/2896] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:06:05 2022] PUT /identity/v3/projects/22037e56af1445c3a88064031d7d4254/users/35e2c522107f4744b8357889a07595eb/roles/efa4ca02b2ff4ad6b14d501d4d79bbea => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:06:05.333948 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11832e-a13b-42e5-b848-cc82b10de1bc None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:05.334238 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11832e-a13b-42e5-b848-cc82b10de1bc None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:05.334516 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c11832e-a13b-42e5-b848-cc82b10de1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:05.354036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-8c11832e-a13b-42e5-b848-cc82b10de1bc None None] MFA Rules not processed for user `e8f45104d2d540658c4bbc32613a9317`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:06:05.356263 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8c11832e-a13b-42e5-b848-cc82b10de1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:05.426256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1429/2897] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:06:05.437069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd70fe-4c6f-4c5e-b476-99c710685c96 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:05.439006 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-52fd70fe-4c6f-4c5e-b476-99c710685c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:05.504654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd70fe-4c6f-4c5e-b476-99c710685c96 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:05.508620 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c3d68d-887e-40b0-875a-2b0ffea38cd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:05.509383 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3d68d-887e-40b0-875a-2b0ffea38cd6 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:05.509743 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3d68d-887e-40b0-875a-2b0ffea38cd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:05.510161 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c3d68d-887e-40b0-875a-2b0ffea38cd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:05.519889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3d68d-887e-40b0-875a-2b0ffea38cd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:05.521473 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c3d68d-887e-40b0-875a-2b0ffea38cd6 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:05.531677 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1469/2898] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:06:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:06:11.764084 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11832e-a13b-42e5-b848-cc82b10de1bc None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:11.766166 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-8c11832e-a13b-42e5-b848-cc82b10de1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:11.825398 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89963ce-577e-4f96-8012-8201a768ad5c None None] REQUEST_METHOD: `POST` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:11.825997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89963ce-577e-4f96-8012-8201a768ad5c None None] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:11.826222 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89963ce-577e-4f96-8012-8201a768ad5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:11.828173 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:06:11.856174 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c11832e-a13b-42e5-b848-cc82b10de1bc None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:11.861708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c19dbd-69bd-4d58-9b92-e55f811b44b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03bc280d0395479981b7c7b08c96001b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:11.862421 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c19dbd-69bd-4d58-9b92-e55f811b44b9 service placement] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:11.862547 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.auth.core [None req-c89963ce-577e-4f96-8012-8201a768ad5c None None] MFA Rules not processed for user `35e2c522107f4744b8357889a07595eb`. Rule list: `[]` (Enabled: `True`). {{(pid=93427) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:06:11.862708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c19dbd-69bd-4d58-9b92-e55f811b44b9 service placement] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:11.863013 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c19dbd-69bd-4d58-9b92-e55f811b44b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:11.863928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c89963ce-577e-4f96-8012-8201a768ad5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:11.864990 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-48c19dbd-69bd-4d58-9b92-e55f811b44b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:11.878130 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c19dbd-69bd-4d58-9b92-e55f811b44b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:11.880191 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c19dbd-69bd-4d58-9b92-e55f811b44b9 service placement] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:11.892708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1430/2899] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:06:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:06:11.939343 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1470/2900] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:06:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:06:11.944870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c19dbd-69bd-4d58-9b92-e55f811b44b9 service placement] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:11.958751 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89963ce-577e-4f96-8012-8201a768ad5c None None] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:11.961698 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c89963ce-577e-4f96-8012-8201a768ad5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:11.962484 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c19dbd-69bd-4d58-9b92-e55f811b44b9 service placement] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:11.972845 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9459d8-159a-4a31-8bc8-fc5bb481b22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:11.973742 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9459d8-159a-4a31-8bc8-fc5bb481b22a admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:11.974256 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9459d8-159a-4a31-8bc8-fc5bb481b22a admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:11.974649 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9459d8-159a-4a31-8bc8-fc5bb481b22a admin admin] PATH_INFO: `/v3/projects` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:11.975390 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9459d8-159a-4a31-8bc8-fc5bb481b22a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:11.977086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89963ce-577e-4f96-8012-8201a768ad5c None None] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:11.977944 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9459d8-159a-4a31-8bc8-fc5bb481b22a admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:11.982045 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3dd65-0c04-4fb8-ae3b-3c6c11aaace6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:11.985091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3dd65-0c04-4fb8-ae3b-3c6c11aaace6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:11.985091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3dd65-0c04-4fb8-ae3b-3c6c11aaace6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:11.985091 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3dd65-0c04-4fb8-ae3b-3c6c11aaace6 admin admin] PATH_INFO: `/v3/users/04af0db8835d427092c076d35b3e67d9` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:11.986236 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3dd65-0c04-4fb8-ae3b-3c6c11aaace6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04af0db8835d427092c076d35b3e67d9)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:11.989114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3dd65-0c04-4fb8-ae3b-3c6c11aaace6 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:12.179836 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1431/2901] 10.222.0.42 () {60 vars in 1153 bytes} [Wed Sep 14 20:06:11 2022] POST /identity/v3/projects => generated 377 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:06:12.181360 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9459d8-159a-4a31-8bc8-fc5bb481b22a admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:12.246899 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9459d8-159a-4a31-8bc8-fc5bb481b22a admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:12.251773 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c6b61-5d3e-441d-928c-2cf4132f308f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:12.252388 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c6b61-5d3e-441d-928c-2cf4132f308f admin admin] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:12.252664 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c6b61-5d3e-441d-928c-2cf4132f308f admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:12.252942 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c6b61-5d3e-441d-928c-2cf4132f308f admin admin] PATH_INFO: `/v3/users` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:12.253634 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c6b61-5d3e-441d-928c-2cf4132f308f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:12.256164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c6b61-5d3e-441d-928c-2cf4132f308f admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:12.333211 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1432/2902] 10.222.0.42 () {60 vars in 1144 bytes} [Wed Sep 14 20:06:12 2022] POST /identity/v3/users => generated 295 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 20:06:12.336897 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c6b61-5d3e-441d-928c-2cf4132f308f admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:12.342332 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d6f3dd65-0c04-4fb8-ae3b-3c6c11aaace6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04af0db8835d427092c076d35b3e67d9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:12.348996 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c6b61-5d3e-441d-928c-2cf4132f308f admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:12.354031 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763c2bb-53ee-499a-bce5-327f15547a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:12.354784 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763c2bb-53ee-499a-bce5-327f15547a91 admin admin] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:12.355071 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763c2bb-53ee-499a-bce5-327f15547a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:12.355333 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7763c2bb-53ee-499a-bce5-327f15547a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:12.355971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763c2bb-53ee-499a-bce5-327f15547a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:12.357590 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7763c2bb-53ee-499a-bce5-327f15547a91 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:12.369687 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1433/2903] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 20:06:12.373917 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763c2bb-53ee-499a-bce5-327f15547a91 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:12.384299 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7763c2bb-53ee-499a-bce5-327f15547a91 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:12.389211 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d2ccb-0b00-40c3-b845-685e2e72f475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:12.389909 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d2ccb-0b00-40c3-b845-685e2e72f475 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:12.390185 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d2ccb-0b00-40c3-b845-685e2e72f475 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:12.390453 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782d2ccb-0b00-40c3-b845-685e2e72f475 admin admin] PATH_INFO: `/v3/projects/35f13f9b7cbf45bea658a19a0a823571/users/138d097d316f4bb2b469e9b44530c92c/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:12.430521 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d2ccb-0b00-40c3-b845-685e2e72f475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f13f9b7cbf45bea658a19a0a823571, user_id=138d097d316f4bb2b469e9b44530c92c, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:12.435228 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782d2ccb-0b00-40c3-b845-685e2e72f475 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:12.435797 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d6f3dd65-0c04-4fb8-ae3b-3c6c11aaace6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04af0db8835d427092c076d35b3e67d9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:12.447691 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:06:12.450110 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d6f3dd65-0c04-4fb8-ae3b-3c6c11aaace6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04af0db8835d427092c076d35b3e67d9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:12.450773 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d6f3dd65-0c04-4fb8-ae3b-3c6c11aaace6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04af0db8835d427092c076d35b3e67d9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:12.451358 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-d6f3dd65-0c04-4fb8-ae3b-3c6c11aaace6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04af0db8835d427092c076d35b3e67d9'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:12.469248 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1471/2904] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:06:11 2022] DELETE /identity/v3/users/04af0db8835d427092c076d35b3e67d9 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:06:12.611198 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1434/2905] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:06:12 2022] PUT /identity/v3/projects/35f13f9b7cbf45bea658a19a0a823571/users/138d097d316f4bb2b469e9b44530c92c/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:06:12.615811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3dd65-0c04-4fb8-ae3b-3c6c11aaace6 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:12.617365 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-d6f3dd65-0c04-4fb8-ae3b-3c6c11aaace6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:12.698518 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3dd65-0c04-4fb8-ae3b-3c6c11aaace6 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:12.758822 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954517f7-87af-4c6f-87af-e0719624a221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:12.759534 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954517f7-87af-4c6f-87af-e0719624a221 admin admin] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:12.759882 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954517f7-87af-4c6f-87af-e0719624a221 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:12.760229 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954517f7-87af-4c6f-87af-e0719624a221 admin admin] PATH_INFO: `/v3/roles` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:12.761020 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954517f7-87af-4c6f-87af-e0719624a221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:12.762731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954517f7-87af-4c6f-87af-e0719624a221 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:12.777499 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1472/2906] 10.222.0.42 () {58 vars in 1122 bytes} [Wed Sep 14 20:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 20:06:12.778799 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d2ccb-0b00-40c3-b845-685e2e72f475 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:12.795064 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954517f7-87af-4c6f-87af-e0719624a221 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:12.796913 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-954517f7-87af-4c6f-87af-e0719624a221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:12.798553 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782d2ccb-0b00-40c3-b845-685e2e72f475 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:12.803353 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4e80e-604a-4c40-997a-e64622c6d7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:12.803974 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4e80e-604a-4c40-997a-e64622c6d7d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:12.804247 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4e80e-604a-4c40-997a-e64622c6d7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:12.804520 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c4e80e-604a-4c40-997a-e64622c6d7d0 admin admin] PATH_INFO: `/v3/projects/35f13f9b7cbf45bea658a19a0a823571/users/138d097d316f4bb2b469e9b44530c92c/roles/70ef12c91aa140af8aaf02fe80527ebb` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:12.806517 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4e80e-604a-4c40-997a-e64622c6d7d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f13f9b7cbf45bea658a19a0a823571, user_id=138d097d316f4bb2b469e9b44530c92c, role_id=70ef12c91aa140af8aaf02fe80527ebb)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:12.809163 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c4e80e-604a-4c40-997a-e64622c6d7d0 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:12.818279 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954517f7-87af-4c6f-87af-e0719624a221 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:12.825716 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1435/2907] 10.222.0.42 () {60 vars in 1483 bytes} [Wed Sep 14 20:06:12 2022] PUT /identity/v3/projects/35f13f9b7cbf45bea658a19a0a823571/users/138d097d316f4bb2b469e9b44530c92c/roles/70ef12c91aa140af8aaf02fe80527ebb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:06:12.900216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb57969-0029-4484-89a7-d5b10378961a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:12.901037 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb57969-0029-4484-89a7-d5b10378961a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:12.901384 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb57969-0029-4484-89a7-d5b10378961a admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:12.901657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb57969-0029-4484-89a7-d5b10378961a admin admin] PATH_INFO: `/v3/projects/2d5c2273800e43a096bce8807c31fbf5` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:12.902860 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb57969-0029-4484-89a7-d5b10378961a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d5c2273800e43a096bce8807c31fbf5)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:12.904813 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb57969-0029-4484-89a7-d5b10378961a admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:13.041997 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bcb57969-0029-4484-89a7-d5b10378961a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d5c2273800e43a096bce8807c31fbf5', 'request_id': 'req-bcb57969-0029-4484-89a7-d5b10378961a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:13.111961 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bcb57969-0029-4484-89a7-d5b10378961a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d5c2273800e43a096bce8807c31fbf5', 'request_id': 'req-bcb57969-0029-4484-89a7-d5b10378961a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:13.120275 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bcb57969-0029-4484-89a7-d5b10378961a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d5c2273800e43a096bce8807c31fbf5', 'request_id': 'req-bcb57969-0029-4484-89a7-d5b10378961a'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:13.133184 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bcb57969-0029-4484-89a7-d5b10378961a admin admin] The token cache is being invalidate because project 2d5c2273800e43a096bce8807c31fbf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:06:13.133538 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bcb57969-0029-4484-89a7-d5b10378961a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:13.134528 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-bcb57969-0029-4484-89a7-d5b10378961a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:13.136643 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1473/2908] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:06:12 2022] DELETE /identity/v3/projects/2d5c2273800e43a096bce8807c31fbf5 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:06:13.139706 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:06:13.140741 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4e80e-604a-4c40-997a-e64622c6d7d0 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:13.142572 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-07c4e80e-604a-4c40-997a-e64622c6d7d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:13.160186 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c4e80e-604a-4c40-997a-e64622c6d7d0 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:13.218192 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:06:13.228751 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3b8f7-96dd-41ee-b302-e7f8663719de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:13.229669 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3b8f7-96dd-41ee-b302-e7f8663719de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:13.229993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3b8f7-96dd-41ee-b302-e7f8663719de admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:13.230270 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3b8f7-96dd-41ee-b302-e7f8663719de admin admin] PATH_INFO: `/v3/users/e024f51143c64b8eb2849762bad968ea` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:13.231814 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3b8f7-96dd-41ee-b302-e7f8663719de admin admin] RBAC: Authorizing `identity:delete_user(user_id=e024f51143c64b8eb2849762bad968ea)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:13.234176 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3b8f7-96dd-41ee-b302-e7f8663719de admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:13.510643 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-57d3b8f7-96dd-41ee-b302-e7f8663719de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e024f51143c64b8eb2849762bad968ea'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:13.533278 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-57d3b8f7-96dd-41ee-b302-e7f8663719de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e024f51143c64b8eb2849762bad968ea'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:13.606380 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-57d3b8f7-96dd-41ee-b302-e7f8663719de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e024f51143c64b8eb2849762bad968ea'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:13.607182 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-57d3b8f7-96dd-41ee-b302-e7f8663719de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e024f51143c64b8eb2849762bad968ea'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:13.617170 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-57d3b8f7-96dd-41ee-b302-e7f8663719de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e024f51143c64b8eb2849762bad968ea'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:13.624991 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1436/2909] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:06:13 2022] DELETE /identity/v3/users/e024f51143c64b8eb2849762bad968ea => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 20:06:13.895235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb57969-0029-4484-89a7-d5b10378961a admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:13.911906 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb57969-0029-4484-89a7-d5b10378961a admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:13.976512 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:13.977077 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:13.977356 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:13.977629 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] PATH_INFO: `/v3/projects/363d06d74a254e9ca1f390ca4c961096` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:13.978965 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=363d06d74a254e9ca1f390ca4c961096)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:13.980874 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:14.123432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '363d06d74a254e9ca1f390ca4c961096', 'request_id': 'req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:14.223478 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '363d06d74a254e9ca1f390ca4c961096', 'request_id': 'req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:14.235433 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '363d06d74a254e9ca1f390ca4c961096', 'request_id': 'req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:14.248844 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] The token cache is being invalidate because project 363d06d74a254e9ca1f390ca4c961096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93427) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:06:14.249130 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:14.250147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:06:14.251855 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1474/2910] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:06:13 2022] DELETE /identity/v3/projects/363d06d74a254e9ca1f390ca4c961096 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:06:18.659409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a953ef9-f10a-44d4-acf5-9e8362d5e25e None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:18.660038 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a953ef9-f10a-44d4-acf5-9e8362d5e25e None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:18.660154 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a953ef9-f10a-44d4-acf5-9e8362d5e25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:18.689494 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-0a953ef9-f10a-44d4-acf5-9e8362d5e25e None None] MFA Rules not processed for user `138d097d316f4bb2b469e9b44530c92c`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:06:18.690684 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-0a953ef9-f10a-44d4-acf5-9e8362d5e25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:18.761718 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1437/2911] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:06:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:06:18.777345 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:18.778775 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:18.831369 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:06:18.855344 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cacd3ad-4cc4-4369-8d31-9ce448ed3560 admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:18.860913 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388dc35-269f-4e2e-8b21-2ccec24a19c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:18.861520 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388dc35-269f-4e2e-8b21-2ccec24a19c4 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:18.861820 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388dc35-269f-4e2e-8b21-2ccec24a19c4 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:18.862092 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388dc35-269f-4e2e-8b21-2ccec24a19c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:18.864344 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6388dc35-269f-4e2e-8b21-2ccec24a19c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:18.923472 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388dc35-269f-4e2e-8b21-2ccec24a19c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:18.925079 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388dc35-269f-4e2e-8b21-2ccec24a19c4 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:18.934560 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1475/2912] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:06:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:06:28.758745 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=63,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:06:29.748992 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=34 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:06:39.904451 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17490c4d-f495-4d13-9360-d4d62822f2c9 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:39.904933 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17490c4d-f495-4d13-9360-d4d62822f2c9 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:39.905063 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17490c4d-f495-4d13-9360-d4d62822f2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:39.924181 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-17490c4d-f495-4d13-9360-d4d62822f2c9 None None] MFA Rules not processed for user `35e2c522107f4744b8357889a07595eb`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:06:39.925366 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-17490c4d-f495-4d13-9360-d4d62822f2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:39.932714 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1438/2913] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:06:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:06:40.239977 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388dc35-269f-4e2e-8b21-2ccec24a19c4 service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:40.241395 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-6388dc35-269f-4e2e-8b21-2ccec24a19c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:40.300229 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388dc35-269f-4e2e-8b21-2ccec24a19c4 service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:40.301701 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d3a2d-965f-4a80-b893-d4bbb86d771f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:40.302327 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d3a2d-965f-4a80-b893-d4bbb86d771f service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:40.302597 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d3a2d-965f-4a80-b893-d4bbb86d771f service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:40.302862 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d3a2d-965f-4a80-b893-d4bbb86d771f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:40.304564 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c66d3a2d-965f-4a80-b893-d4bbb86d771f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:40.314523 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d3a2d-965f-4a80-b893-d4bbb86d771f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:40.315863 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d3a2d-965f-4a80-b893-d4bbb86d771f service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:40.323710 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1476/2914] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:06:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:06:41.096596 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17490c4d-f495-4d13-9360-d4d62822f2c9 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:41.098207 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-17490c4d-f495-4d13-9360-d4d62822f2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:41.167971 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17490c4d-f495-4d13-9360-d4d62822f2c9 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:41.172925 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961cb89-0033-4a18-8532-02db3b63d9b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03bc280d0395479981b7c7b08c96001b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:41.173527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961cb89-0033-4a18-8532-02db3b63d9b5 service placement] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:41.173832 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961cb89-0033-4a18-8532-02db3b63d9b5 service placement] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:41.174096 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961cb89-0033-4a18-8532-02db3b63d9b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:41.175767 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-c961cb89-0033-4a18-8532-02db3b63d9b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:41.187561 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961cb89-0033-4a18-8532-02db3b63d9b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:41.189164 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961cb89-0033-4a18-8532-02db3b63d9b5 service placement] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:41.199600 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1439/2915] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:06:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:06:44.471941 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d3a2d-965f-4a80-b893-d4bbb86d771f service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:44.482714 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d3a2d-965f-4a80-b893-d4bbb86d771f service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:44.487082 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c04c20-19eb-42ac-b656-78a0ef3923a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:44.488114 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04c20-19eb-42ac-b656-78a0ef3923a9 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:44.488696 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04c20-19eb-42ac-b656-78a0ef3923a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:44.489068 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c04c20-19eb-42ac-b656-78a0ef3923a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:44.490963 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f2c04c20-19eb-42ac-b656-78a0ef3923a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:44.503286 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c04c20-19eb-42ac-b656-78a0ef3923a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:44.505086 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c04c20-19eb-42ac-b656-78a0ef3923a9 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:44.513411 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1477/2916] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:06:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:06:45.288575 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961cb89-0033-4a18-8532-02db3b63d9b5 service placement] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:45.302049 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961cb89-0033-4a18-8532-02db3b63d9b5 service placement] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:45.307511 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62073184-42f6-48d3-b4c8-d62512a1af6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:45.308109 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62073184-42f6-48d3-b4c8-d62512a1af6e service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:45.308433 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62073184-42f6-48d3-b4c8-d62512a1af6e service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:45.308708 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62073184-42f6-48d3-b4c8-d62512a1af6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:45.310506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-62073184-42f6-48d3-b4c8-d62512a1af6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:45.323527 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62073184-42f6-48d3-b4c8-d62512a1af6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:45.325101 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62073184-42f6-48d3-b4c8-d62512a1af6e service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:45.336294 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1440/2917] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:06:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:06:54.509615 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=32 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:06:55.332248 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=41 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:06:56.873657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c04c20-19eb-42ac-b656-78a0ef3923a9 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:06:56.875336 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-f2c04c20-19eb-42ac-b656-78a0ef3923a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:56.890903 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c04c20-19eb-42ac-b656-78a0ef3923a9 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:06:56.896371 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611f142-7dce-4c1e-83b1-10541f5a7c62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:06:56.897700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611f142-7dce-4c1e-83b1-10541f5a7c62 service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:06:56.897700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611f142-7dce-4c1e-83b1-10541f5a7c62 service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:06:56.897700 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611f142-7dce-4c1e-83b1-10541f5a7c62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:06:56.899287 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-8611f142-7dce-4c1e-83b1-10541f5a7c62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:06:56.911678 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611f142-7dce-4c1e-83b1-10541f5a7c62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:06:56.913232 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611f142-7dce-4c1e-83b1-10541f5a7c62 service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:06:56.924235 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1478/2918] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:06:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:07:00.038495 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62073184-42f6-48d3-b4c8-d62512a1af6e service nova] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:07:00.048415 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62073184-42f6-48d3-b4c8-d62512a1af6e service nova] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:07:00.051966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac3762-f733-4028-a290-91088dc39ec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:07:00.052551 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac3762-f733-4028-a290-91088dc39ec9 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:07:00.052824 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac3762-f733-4028-a290-91088dc39ec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:07:00.053094 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac3762-f733-4028-a290-91088dc39ec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:07:00.061817 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac3762-f733-4028-a290-91088dc39ec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:07:00.063215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac3762-f733-4028-a290-91088dc39ec9 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:07:00.072251 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1441/2919] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:07:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:07:06.919807 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=6 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:07:10.068250 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=6 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:08:52.597605 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611f142-7dce-4c1e-83b1-10541f5a7c62 service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:08:52.609206 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611f142-7dce-4c1e-83b1-10541f5a7c62 service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:08:52.614780 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa5486-70aa-4ad3-8a38-61b13cb2f549 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03bc280d0395479981b7c7b08c96001b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:08:52.615376 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa5486-70aa-4ad3-8a38-61b13cb2f549 service placement] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:08:52.615654 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa5486-70aa-4ad3-8a38-61b13cb2f549 service placement] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:08:52.615928 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa5486-70aa-4ad3-8a38-61b13cb2f549 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:08:52.617624 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7efa5486-70aa-4ad3-8a38-61b13cb2f549 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:08:52.630107 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa5486-70aa-4ad3-8a38-61b13cb2f549 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:08:52.631731 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa5486-70aa-4ad3-8a38-61b13cb2f549 service placement] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:08:52.641727 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1479/2920] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:08:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:09:02.637291 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=6 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:09:04.346782 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f5535-04e3-4038-bc37-0a8a3cbc23a2 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:09:04.347293 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f5535-04e3-4038-bc37-0a8a3cbc23a2 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:09:04.347407 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f5535-04e3-4038-bc37-0a8a3cbc23a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:09:04.367472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-4c8f5535-04e3-4038-bc37-0a8a3cbc23a2 None None] MFA Rules not processed for user `35e2c522107f4744b8357889a07595eb`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:09:04.368679 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4c8f5535-04e3-4038-bc37-0a8a3cbc23a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:09:04.375552 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1442/2921] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:09:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:09:04.691653 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa5486-70aa-4ad3-8a38-61b13cb2f549 service placement] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:09:04.693646 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-7efa5486-70aa-4ad3-8a38-61b13cb2f549 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:09:04.704770 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa5486-70aa-4ad3-8a38-61b13cb2f549 service placement] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:09:04.708326 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60164a65-373a-42ea-8968-24594c3c98c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:09:04.708902 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60164a65-373a-42ea-8968-24594c3c98c9 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:09:04.709173 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60164a65-373a-42ea-8968-24594c3c98c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:09:04.709462 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60164a65-373a-42ea-8968-24594c3c98c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:09:04.711084 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-60164a65-373a-42ea-8968-24594c3c98c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:09:04.720328 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60164a65-373a-42ea-8968-24594c3c98c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:09:04.721717 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60164a65-373a-42ea-8968-24594c3c98c9 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:09:04.730129 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1480/2922] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:09:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:09:14.361746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=3 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:09:14.726524 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=6 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:11:11.632472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2a431-2858-4b7d-a6e2-ed2b38206508 None None] REQUEST_METHOD: `POST` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:11:11.633031 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2a431-2858-4b7d-a6e2-ed2b38206508 None None] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:11:11.633031 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2a431-2858-4b7d-a6e2-ed2b38206508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:11:11.652178 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.auth.core [None req-39e2a431-2858-4b7d-a6e2-ed2b38206508 None None] MFA Rules not processed for user `35e2c522107f4744b8357889a07595eb`. Rule list: `[]` (Enabled: `True`). {{(pid=93426) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 20:11:11.653377 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-39e2a431-2858-4b7d-a6e2-ed2b38206508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:11:11.660157 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1443/2923] 10.222.0.42 () {58 vars in 958 bytes} [Wed Sep 14 20:11:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 20:11:11.702017 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60164a65-373a-42ea-8968-24594c3c98c9 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:11:11.703428 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-60164a65-373a-42ea-8968-24594c3c98c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:11:11.758351 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60164a65-373a-42ea-8968-24594c3c98c9 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:11:11.762070 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7aac5d-7fc3-4d93-a849-cd39781c5855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a2c9eec08b142138569043e3b34c6a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:11:11.762695 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7aac5d-7fc3-4d93-a849-cd39781c5855 service cinder] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:11:11.763027 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7aac5d-7fc3-4d93-a849-cd39781c5855 service cinder] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:11:11.763882 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7aac5d-7fc3-4d93-a849-cd39781c5855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:11:11.765657 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-0a7aac5d-7fc3-4d93-a849-cd39781c5855 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:11:11.779483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7aac5d-7fc3-4d93-a849-cd39781c5855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:11:11.780861 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7aac5d-7fc3-4d93-a849-cd39781c5855 service cinder] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:11:11.791889 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1481/2924] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:11:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:11:12.754171 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2a431-2858-4b7d-a6e2-ed2b38206508 None None] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:11:12.764502 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2a431-2858-4b7d-a6e2-ed2b38206508 None None] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:11:12.768258 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333caaf-6f36-4518-aa41-bdd7e5ba5023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:11:12.768993 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333caaf-6f36-4518-aa41-bdd7e5ba5023 service neutron] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:11:12.769403 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333caaf-6f36-4518-aa41-bdd7e5ba5023 service neutron] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:11:12.769696 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333caaf-6f36-4518-aa41-bdd7e5ba5023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:11:12.771958 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-1333caaf-6f36-4518-aa41-bdd7e5ba5023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:11:12.787809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333caaf-6f36-4518-aa41-bdd7e5ba5023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:11:12.787809 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333caaf-6f36-4518-aa41-bdd7e5ba5023 service neutron] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:11:12.799815 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1444/2925] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:11:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:11:18.684556 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7aac5d-7fc3-4d93-a849-cd39781c5855 service cinder] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:11:18.695690 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7aac5d-7fc3-4d93-a849-cd39781c5855 service cinder] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:11:18.701177 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2afea5-f5d0-41ad-84cf-aaf867626f3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:11:18.701758 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2afea5-f5d0-41ad-84cf-aaf867626f3d service nova] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:11:18.702069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2afea5-f5d0-41ad-84cf-aaf867626f3d service nova] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:11:18.702335 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2afea5-f5d0-41ad-84cf-aaf867626f3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:11:18.709849 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2afea5-f5d0-41ad-84cf-aaf867626f3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:11:18.711421 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2afea5-f5d0-41ad-84cf-aaf867626f3d service nova] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:11:18.721069 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1482/2926] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:11:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:11:22.794868 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=9 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:11:28.716267 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=32 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:11:32.592236 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333caaf-6f36-4518-aa41-bdd7e5ba5023 service neutron] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:11:32.603992 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333caaf-6f36-4518-aa41-bdd7e5ba5023 service neutron] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:11:32.609763 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26d31f-b472-4c6c-814c-90ff17f1acf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:11:32.610399 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26d31f-b472-4c6c-814c-90ff17f1acf4 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:11:32.610671 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26d31f-b472-4c6c-814c-90ff17f1acf4 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:11:32.610939 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26d31f-b472-4c6c-814c-90ff17f1acf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:11:32.612635 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-4e26d31f-b472-4c6c-814c-90ff17f1acf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:11:32.625361 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26d31f-b472-4c6c-814c-90ff17f1acf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:11:32.626973 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26d31f-b472-4c6c-814c-90ff17f1acf4 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:11:32.637485 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1445/2927] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:11:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:11:42.632926 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=6 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:11:54.252993 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2afea5-f5d0-41ad-84cf-aaf867626f3d service nova] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:11:54.262440 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2afea5-f5d0-41ad-84cf-aaf867626f3d service nova] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:11:54.266085 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69f774-b27e-4b8b-b3bf-78a8cdb4bf75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:11:54.266699 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69f774-b27e-4b8b-b3bf-78a8cdb4bf75 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:11:54.266971 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69f774-b27e-4b8b-b3bf-78a8cdb4bf75 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:11:54.267250 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab69f774-b27e-4b8b-b3bf-78a8cdb4bf75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:11:54.274387 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69f774-b27e-4b8b-b3bf-78a8cdb4bf75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:11:54.275785 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab69f774-b27e-4b8b-b3bf-78a8cdb4bf75 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:11:54.284331 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1483/2928] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:11:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:12:04.280074 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=6 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:13:15.561438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26d31f-b472-4c6c-814c-90ff17f1acf4 service nova] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:13:15.605306 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26d31f-b472-4c6c-814c-90ff17f1acf4 service nova] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:13:15.610870 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a55a5-b18d-4b23-a520-ace5eed41787 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a4284b59f804dc59674d621d87634bc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'ResellerAdmin', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:13:15.611470 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a55a5-b18d-4b23-a520-ace5eed41787 service nova] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:13:15.611746 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a55a5-b18d-4b23-a520-ace5eed41787 service nova] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:13:15.612017 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a55a5-b18d-4b23-a520-ace5eed41787 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:13:15.621618 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a55a5-b18d-4b23-a520-ace5eed41787 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:13:15.623221 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a55a5-b18d-4b23-a520-ace5eed41787 service nova] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:13:15.634472 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1446/2929] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:13:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:13:25.629262 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=20 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:13:27.171330 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69f774-b27e-4b8b-b3bf-78a8cdb4bf75 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:13:27.208483 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab69f774-b27e-4b8b-b3bf-78a8cdb4bf75 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:13:27.212241 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d9e72-8fc5-486e-bd49-7e5e57d63079 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7103ff9544524393a8c7175998dcae3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:13:27.213616 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d9e72-8fc5-486e-bd49-7e5e57d63079 service neutron] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:13:27.213781 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d9e72-8fc5-486e-bd49-7e5e57d63079 service neutron] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:13:27.214078 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d9e72-8fc5-486e-bd49-7e5e57d63079 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:13:27.215827 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fb3d9e72-8fc5-486e-bd49-7e5e57d63079 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:13:27.307425 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d9e72-8fc5-486e-bd49-7e5e57d63079 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:13:27.308840 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d9e72-8fc5-486e-bd49-7e5e57d63079 service neutron] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:13:27.327660 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1484/2930] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:13:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 20:13:30.965779 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a55a5-b18d-4b23-a520-ace5eed41787 service nova] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:13:30.977594 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a55a5-b18d-4b23-a520-ace5eed41787 service nova] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:13:30.982837 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54247d-95d2-4df1-ad52-e0b1adf4ae24 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03bc280d0395479981b7c7b08c96001b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:13:30.983500 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54247d-95d2-4df1-ad52-e0b1adf4ae24 service placement] REQUEST_METHOD: `GET` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:13:30.983841 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54247d-95d2-4df1-ad52-e0b1adf4ae24 service placement] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:13:30.984177 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54247d-95d2-4df1-ad52-e0b1adf4ae24 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:13:30.986112 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-9f54247d-95d2-4df1-ad52-e0b1adf4ae24 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:13:30.998703 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54247d-95d2-4df1-ad52-e0b1adf4ae24 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:13:31.000270 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54247d-95d2-4df1-ad52-e0b1adf4ae24 service placement] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:13:31.010524 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1447/2931] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:13:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:13:34.472432 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d9e72-8fc5-486e-bd49-7e5e57d63079 service neutron] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:13:34.473890 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-fb3d9e72-8fc5-486e-bd49-7e5e57d63079 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:13:34.485563 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d9e72-8fc5-486e-bd49-7e5e57d63079 service neutron] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:13:34.490326 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23660aca-d938-40ee-88b0-a8cc1ad62a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:13:34.490943 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23660aca-d938-40ee-88b0-a8cc1ad62a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:13:34.491216 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23660aca-d938-40ee-88b0-a8cc1ad62a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:13:34.491480 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23660aca-d938-40ee-88b0-a8cc1ad62a8d admin admin] PATH_INFO: `/v3/users/35e2c522107f4744b8357889a07595eb` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:13:34.492568 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23660aca-d938-40ee-88b0-a8cc1ad62a8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e2c522107f4744b8357889a07595eb)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:13:34.495000 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23660aca-d938-40ee-88b0-a8cc1ad62a8d admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:13:34.683288 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-23660aca-d938-40ee-88b0-a8cc1ad62a8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e2c522107f4744b8357889a07595eb'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:34.755234 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-23660aca-d938-40ee-88b0-a8cc1ad62a8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e2c522107f4744b8357889a07595eb'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:34.767038 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-23660aca-d938-40ee-88b0-a8cc1ad62a8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e2c522107f4744b8357889a07595eb'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:34.767665 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-23660aca-d938-40ee-88b0-a8cc1ad62a8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e2c522107f4744b8357889a07595eb'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:34.768262 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-23660aca-d938-40ee-88b0-a8cc1ad62a8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e2c522107f4744b8357889a07595eb'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:34.780118 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1485/2932] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:13:34 2022] DELETE /identity/v3/users/35e2c522107f4744b8357889a07595eb => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:13:35.021950 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54247d-95d2-4df1-ad52-e0b1adf4ae24 service placement] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:13:35.033169 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54247d-95d2-4df1-ad52-e0b1adf4ae24 service placement] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:13:35.090840 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:13:35.091504 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:13:35.091780 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:13:35.092051 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] PATH_INFO: `/v3/projects/22037e56af1445c3a88064031d7d4254` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:13:35.093192 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22037e56af1445c3a88064031d7d4254)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:13:35.095381 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:13:35.203506 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22037e56af1445c3a88064031d7d4254', 'request_id': 'req-d608908b-580f-413f-970e-2993cf9a8109'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:35.209036 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22037e56af1445c3a88064031d7d4254', 'request_id': 'req-d608908b-580f-413f-970e-2993cf9a8109'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:35.215692 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22037e56af1445c3a88064031d7d4254', 'request_id': 'req-d608908b-580f-413f-970e-2993cf9a8109'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:35.313319 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] The token cache is being invalidate because project 22037e56af1445c3a88064031d7d4254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:13:35.313628 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:35.314314 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:35.316295 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1448/2933] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:13:35 2022] DELETE /identity/v3/projects/22037e56af1445c3a88064031d7d4254 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:13:35.320237 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23660aca-d938-40ee-88b0-a8cc1ad62a8d admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:13:35.321681 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-23660aca-d938-40ee-88b0-a8cc1ad62a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:13:35.329835 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:13:35.335655 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23660aca-d938-40ee-88b0-a8cc1ad62a8d admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:13:35.390740 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ca6cf-51a4-45d9-8359-5934b7e8086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:13:35.391366 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ca6cf-51a4-45d9-8359-5934b7e8086e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:13:35.391637 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ca6cf-51a4-45d9-8359-5934b7e8086e admin admin] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:13:35.391904 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ca6cf-51a4-45d9-8359-5934b7e8086e admin admin] PATH_INFO: `/v3/users/138d097d316f4bb2b469e9b44530c92c` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:13:35.393033 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ca6cf-51a4-45d9-8359-5934b7e8086e admin admin] RBAC: Authorizing `identity:delete_user(user_id=138d097d316f4bb2b469e9b44530c92c)` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:13:35.394964 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30ca6cf-51a4-45d9-8359-5934b7e8086e admin admin] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:13:35.585435 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c30ca6cf-51a4-45d9-8359-5934b7e8086e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '138d097d316f4bb2b469e9b44530c92c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:35.651930 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c30ca6cf-51a4-45d9-8359-5934b7e8086e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '138d097d316f4bb2b469e9b44530c92c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:35.663811 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c30ca6cf-51a4-45d9-8359-5934b7e8086e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '138d097d316f4bb2b469e9b44530c92c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:35.664443 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c30ca6cf-51a4-45d9-8359-5934b7e8086e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '138d097d316f4bb2b469e9b44530c92c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:35.665061 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.notifications [None req-c30ca6cf-51a4-45d9-8359-5934b7e8086e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '138d097d316f4bb2b469e9b44530c92c'} {{(pid=93427) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:35.677940 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1486/2934] 10.222.0.42 () {58 vars in 1224 bytes} [Wed Sep 14 20:13:35 2022] DELETE /identity/v3/users/138d097d316f4bb2b469e9b44530c92c => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:13:35.930409 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] Authenticating user token {{(pid=93426) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:13:35.931824 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.fernet_utils [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93426) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:13:35.945651 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608908b-580f-413f-970e-2993cf9a8109 admin admin] Validating token access rules against request {{(pid=93426) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:13:36.003257 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097ce6b1-259f-4c65-bc23-b900b7c127de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8f45104d2d540658c4bbc32613a9317', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2a8f053b53ae4aff97a668669e77d899', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93426) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:13:36.003886 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ce6b1-259f-4c65-bc23-b900b7c127de admin admin] REQUEST_METHOD: `DELETE` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:13:36.004165 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ce6b1-259f-4c65-bc23-b900b7c127de admin admin] SCRIPT_NAME: `/identity` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:13:36.004438 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097ce6b1-259f-4c65-bc23-b900b7c127de admin admin] PATH_INFO: `/v3/projects/35f13f9b7cbf45bea658a19a0a823571` {{(pid=93426) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:13:36.005560 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ce6b1-259f-4c65-bc23-b900b7c127de admin admin] RBAC: Authorizing `identity:delete_project(project_id=35f13f9b7cbf45bea658a19a0a823571)` {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:13:36.007512 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097ce6b1-259f-4c65-bc23-b900b7c127de admin admin] RBAC: Authorization granted {{(pid=93426) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:13:36.029260 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:13:36.103463 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-097ce6b1-259f-4c65-bc23-b900b7c127de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35f13f9b7cbf45bea658a19a0a823571', 'request_id': 'req-097ce6b1-259f-4c65-bc23-b900b7c127de'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:36.108963 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-097ce6b1-259f-4c65-bc23-b900b7c127de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35f13f9b7cbf45bea658a19a0a823571', 'request_id': 'req-097ce6b1-259f-4c65-bc23-b900b7c127de'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:36.115966 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-097ce6b1-259f-4c65-bc23-b900b7c127de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35f13f9b7cbf45bea658a19a0a823571', 'request_id': 'req-097ce6b1-259f-4c65-bc23-b900b7c127de'} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:36.179370 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-097ce6b1-259f-4c65-bc23-b900b7c127de admin admin] The token cache is being invalidate because project 35f13f9b7cbf45bea658a19a0a823571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93426) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 14 20:13:36.179597 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-097ce6b1-259f-4c65-bc23-b900b7c127de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:36.180215 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG keystone.notifications [None req-097ce6b1-259f-4c65-bc23-b900b7c127de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93426) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 14 20:13:36.181699 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: [pid: 93426|app: 0|req: 1449/2935] 10.222.0.42 () {58 vars in 1233 bytes} [Wed Sep 14 20:13:35 2022] DELETE /identity/v3/projects/35f13f9b7cbf45bea658a19a0a823571 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 20:13:45.674543 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=52,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:13:46.176239 dv-cinder-839768-8-iscsi devstack@keystone.service[93426]: DEBUG dbcounter [-] [93426] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93426) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 20:13:58.557868 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ca6cf-51a4-45d9-8359-5934b7e8086e admin admin] Authenticating user token {{(pid=93427) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 20:13:58.559277 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c30ca6cf-51a4-45d9-8359-5934b7e8086e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:13:58.646614 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30ca6cf-51a4-45d9-8359-5934b7e8086e admin admin] Validating token access rules against request {{(pid=93427) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 20:13:58.703577 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66a2c1b-3fc3-44d5-8540-9facd7d12fb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03bc280d0395479981b7c7b08c96001b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6bfc6c1c443849038af35577daf34088', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93427) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 20:13:58.704171 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a2c1b-3fc3-44d5-8540-9facd7d12fb3 service placement] REQUEST_METHOD: `GET` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 20:13:58.704448 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a2c1b-3fc3-44d5-8540-9facd7d12fb3 service placement] SCRIPT_NAME: `/identity` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 20:13:58.704719 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a2c1b-3fc3-44d5-8540-9facd7d12fb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93427) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 20:13:58.706455 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.fernet_utils [None req-c66a2c1b-3fc3-44d5-8540-9facd7d12fb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93427) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 20:13:58.717300 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a2c1b-3fc3-44d5-8540-9facd7d12fb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 20:13:58.719147 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66a2c1b-3fc3-44d5-8540-9facd7d12fb3 service placement] RBAC: Authorization granted {{(pid=93427) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 20:13:58.729348 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: [pid: 93427|app: 0|req: 1487/2936] 10.222.0.42 () {60 vars in 1365 bytes} [Wed Sep 14 20:13:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 20:14:08.724641 dv-cinder-839768-8-iscsi devstack@keystone.service[93427]: DEBUG dbcounter [-] [93427] Writing DB stats keystone:SELECT=67 {{(pid=93427) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}